Sr. Databricks Engineer

Job Description

Job Description

Job Title : Sr. Databricks Engineer

Location :   Dallas, TX

Duration :   FTE

Schedule : Hybrid 2 days onsite (Tue / Thurs)

About the job :

Seeking a hands-on Sr. Databricks Data Engineer to design, develop, and optimize data pipelines and analytics solutions. The ideal candidate will have strong experience in data engineering, ETL development, and production support, ensuring reliable, scalable, and high-performing data operations within Azure environment and can work in a fast-paced environment. Knowledge of insurance domain and Power BI is a plus but not mandatory.

Key Responsibilities :

Development

• Design, develop, and deploy scalable ETL / ELT data pipelines using Apache Spark, PySpark, and Databricks.
• Develop and optimize SQL queries for data transformation and analysis.
• Collaborate with product owners, data architects and analysts to build data models, delta lake structures, and data workflows.
• Collaborate with data analysts and business teams to deliver actionable insights.
• Build job orchestration and monitoring solutions
• Ensure data quality, performance, and reliability across workflows.
• Develop and maintain CI / CD pipelines for Databricks notebooks, jobs, and workflows.
• Work with cloud-based data platforms (Azure preferred).

Production Support

Required Skills & Experience :

Nice-to-Have :