Data Engineer - Databricks

A company is looking for a Technical Consultant specializing in Databricks.

Key Responsibilities

Design, develop, and maintain ETL / ELT pipelines and Databricks notebooks for data ingestion, transformation, and reporting

Implement and optimize Spark-based workflows using Databricks Workflows, Delta Lake, and Unity Catalog

Conduct data reconciliation across source, staging, and target systems to ensure accuracy and completeness

Required Qualifications

Minimum 2 years of experience with Databricks, Spark, Python, SQL, and Unity Catalog

Strong proficiency in SQL, including joins, CTEs, and query optimization

Solid experience with PySpark and Python scripting for data manipulation

Familiarity with medallion architecture and data warehouse concepts

Experience validating flat files including delimiter handling and schema enforcement