Job Description
Job Description
Job Title :
Data Engineer Databricks & Azure
Location :
Columbus, OH preferred / Remote (US or Canada)
Experience Required :
712 years
Compensation :
$150K USD / CAD $170K$190K
Job Type : Full-Time
Position Overview :
We are seeking a skilled Data Engineer to join our team. The successful candidate will be responsible for development and optimization of data pipelines, implementing robust data checks, and ensuring the accuracy and integrity of data flows. This role is critical in supporting data-driven decision-making processes, especially in the context of our insurance-focused business operations.
Responsibilities :
- Collaborate with data analysts, reporting team and business advisors to gather requirements and define data models
- Develop and maintain scalable and efficient data pipelines
- Implement robust data checks and validate large datasets
- Monitor data jobs and troubleshoot issues
- Review and audit data processes for compliance
- Work within Agile methodologies including scrum and sprint planning
Qualifications :
Bachelors degree in Computer Science, IT, or related field712 years of experience in Data Engineering with Databricks & CloudStrong proficiency in PySpark, Python, SQLExperience in data modeling, ETL / ELT pipeline development, and automationHands-on experience with Azure Data Factory, Azure Databricks, Azure Data LakeExperience with Delta Lake, Delta Live Tables, Autoloader & Unity CatalogPreferred : Knowledge of insurance industry data requirementsStrong analytical and communication skillsThis is a remote position.