Data Engineer - Python

Seeking a Specialist with 5 to 7 years of experience in designing and implementing data engineering solutions using Python and Azure Data Factory within the Azure Data Factory ADF skill family

Job Description

Design develop and maintain scalable data pipelines and ETL processes using Python and Azure Data Factory Collaborate with source data application teams and product owners to understand data requirements and deliver analytics solutions Utilize Azure Data Factory and Azure Data Lake for data ingestion storage and processing Implement data migration and transformation workflows leveraging Azure cloud services Ensure data quality performance tuning and optimization of data pipelines Support continuous integration and continuous deployment CICD pipelines for data engineering projects Apply best practices for data governance privacy and security within Azure environments Stay updated with emerging Azure technologies and recommend improvements to existing data solutions

Roles and Responsibilities

Lead the design and implementation of batch and streaming data pipelines using Azure Data Factory and related Azure services Provide technical leadership and mentorship to team members on Python programming and Azure Data Factory best practices Collaborate with DevOps teams to manage deployment monitoring and support of data engineering solutions Troubleshoot and resolve data pipeline issues ensuring high availability and reliability Participate in architectural discussions and contribute to the development of coding standards and security guidelines Engage with crossfunctional teams to align data engineering efforts with business objectives Drive automation of data workflows and improve operational efficiency Support production environments and provide timely incident response and root cause analysis

Skills

Mandatory Skills : Python, Azure Data Factory

Good to Have Skills : AZURE DATA LAKE