Senior Databricks Data Engineer (Spark, Python)

Minimum years of experience : - years

Job Details :

Must Have Skills : Databricks, Spark, Python

Nice to have skills : Hadoop, Hive

Detailed Job Description

• years of experience in data engineering.
• Design, develop, and maintain scalable and robust data pipelines on Databricks.
• Optimize and troubleshoot existing data pipelines for performance and reliability.
• Proficiency with Databricks and Spark.
• Implement and optimize Spark jobs, data transformations, and data processing workflows in Databricks.
• Strong SQL skills and experience with relational databases.
• Experience with big data technologies.
• Ensure data quality and integrity across various data sources.
• Implement data security and compliance best practices.
• Monitor data pipeline performance and conduct necessary maintenance and updates.
• Document data pipeline processes and technical specifications.