Senior Data Engineer (With Python Experience)

Job Description

We are seeking a skilled Senior Data Engineer with a solid background in data engineering and software engineering and exposure to machine learning. The ideal candidate will work with Polars as the primary data frame library and contribute to building testing and optimizing data-driven solutions.

Key Responsibilities

• Develop robust data transformations using Python and Polars for efficient processing and analytics tasks.
• Hands-on experience in data engineering
• Write clean maintainable and efficient code with a strong emphasis on software engineering principles.
• Gather data from diverse sources transforming it efficiently and delivering clean reliable datasets and reports that support critical business decision-making.
• Implement and maintain unit testing and end-to-end testing.
• Utilize Docker for testing and development environments.
• Manage data infrastructure on GCP
• Apply a foundational understanding of machine learning concepts in project work.
• Contribute to projects requiring some Ruby coding support.

Requirements

Key Skills

Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala

Employment Type : Full Time

Experience : years

Vacancy : 1