Senior Data Engineer (With Python Experience)

Job Description

We are seeking a skilled Senior Data Engineer with a solid background in data engineering and software engineering and exposure to machine learning. The ideal candidate will work with Polars as the primary data frame library and contribute to building, testing, and optimizing data-driven solutions.

Key Responsibilities

Develop robust data transformations using Python and Polars for efficient processing and analytics tasks.

Hands-on experience in data engineering

Write clean, maintainable, and efficient code with a strong emphasis on software engineering principles.

Gather data from diverse sources, transforming it efficiently, and delivering clean, reliable datasets and reports that support critical business decision-making.

Implement and maintain unit testing and end-to-end testing.

Utilize Docker for testing and development environments.

Manage data infrastructure on GCP

Apply a foundational understanding of machine learning concepts in project work.

Contribute to projects requiring some Ruby coding support.

Requirements

6+ years of experience in data engineering

Strong proficiency in Python.

Hands-on experience in data engineering, including designing, building, and optimizing data pipelines, ETL processes, and workflows.

Solid software engineering skills (design, architecture, and best practices).

Experience working with Polars (or similar data frame libraries).

Knowledge of unit testing and end-to-end testing frameworks.

Hands-on experience with Docker for testing purposes.

Exposure to GCP services.

Basic understanding of machine learning concepts.

Good to have some experience with Ruby.