Advance Data Engineer

Job Description

• Understanding current architecture and design principles of Databricks echo system
• Handson experience with Managing and securing data assets in Databricks using Unity Catalog.
• Understanding Monitoring and observability using Datadog.
• Handson experience with Python / Pyspark scripts for data processing and analysis.
• Deep understanding of the datasets being used.
• Experience of Version control with GitHub and automating workflows using GitHub Actions.
• Experience with DevOps for CI / CD pipeline configurations.
• Knowledge of current AI coding assistants like GitHub Copilot and Databricks Assistant being used

Good understanding of transactional and dimensional data models