Databricks Engineer - Commercial Insurance - Fulltime
Acunor IncColumbus, OH, United States4 days ago
Job type
- Full-time
- Quick Apply
Job descriptionCollaborate with business analysts and reporting teams to define logical and physical data models supporting analytical and operational needs. Implement data warehousing and lakehouse solutions using Azure Data Lake and Delta Lake . Optimize data structures for query performance, cost efficiency, and reusability. Design and implement robust data quality checks and validation mechanisms to maintain integrity across sources and transformations. Automate repetitive processes using scripts, parameterized pipelines, and reusable frameworks. Conduct periodic audits and compliance checks aligned with governance policies. Contribute to metadata management, documentation, and lineage tracking. 7 12 years of experience in Data Engineering with proven expertise in Databricks and Azure Cloud ecosystems. Strong hands-on experience in PySpark , Python , and SQL for data transformation, validation, and performance tuning. Solid understanding of Delta Lake architecture , ETL / ELT frameworks , and data warehousing principles. Proficiency with Azure services including Data Factory (ADF) , Data Lake (ADLS) , and Databricks Notebooks . Experience with Delta Live Tables , Unity Catalog , and Autoloader for batch and streaming data processing. Strong background in data modeling , performance optimization , and automation scripting . Familiarity with Agile methodologies and DevOps-based deployment practices (Git, CI / CD preferred). Strong analytical, communication, and problem-solving skills to collaborate effectively across diverse teams. Preferred : Exposure to insurance, healthcare, or financial services data ecosystems. Experience in data migration projects (on-prem to cloud or multi-cloud). Familiarity with Delta Sharing , Databricks SQL Warehouses , or MLflow for advanced use cases. Experience with data cataloging, lineage, or quality frameworks such as Purview, Collibra, or Great Expectations. Exposure to BI / reporting tools like Power BI or Tableau for end-to-end integration understanding.
ob Title : Data Engineer (Databricks + Azure)
Client : One of our Consulting Clients (Global Analytics & Digital Transformation Firm)
Location : Columbus, OH (Remote / Hybrid)
Duration : Full-Time
About the Role
We are seeking a highly skilled Data Engineer with deep expertise in Databricks and Azure Cloud to join a decision analytics and data engineering team within one of our global consulting clients. The role involves building, optimizing, and maintaining large-scale data pipelines that fuel enterprise analytics, reporting, and AI-driven insights-primarily supporting clients in the insurance and financial services domains .
Key Responsibilities
Data Pipeline Development & Optimization
- Design, build, and enhance ETL / ELT data pipelines using Azure Data Factory , Databricks (PySpark, SQL, Python) , and related services.
- Develop and manage Delta Live Tables , Autoloader , and Unity Catalog within the Databricks ecosystem for structured, incremental data processing.
- Implement data ingestion, transformation, and validation frameworks that ensure high performance, scalability, and reliability.
- Monitor data pipelines, troubleshoot issues, and ensure optimal system performance and SLA adherence.
Data Modeling & Architecture
Data Quality, Governance & Automation
Required Skills & Experience
Nice to Have
Create a job alert for this search
Commercial Insurance • Columbus, OH, United States
Related jobs
Join our team as we reimagine and modernize the core of Relativity's data architecture.You'll play a pivotal role in transforming the Document Domain - a foundational component of our platform-into...Show moreLast updated: 30+ days ago
A company is looking for a Data Science Engineer to implement data science and machine learning solutions.Key Responsibilities Collaborate with teams to understand business requirements and impro...Show moreLast updated: 30+ days ago A company is looking for a Software Engineer, Molecular Data.Key Responsibilities Build, maintain, and improve statistical data pipelines for the computational platform Design, develop, and depl...Show moreLast updated: 30+ days ago Key Responsibilities Design and architect Elasticsearch ecosystems, including cluster sizing Manage data through indices, templates, data streams, and data lifecycle / backup policies Troubleshoo...Show moreLast updated: 30+ days ago A company is looking for a Principal Data Engineer with extensive experience in high-transaction e-commerce environments.
Key Responsibilities Develop and maintain ETL / ELT pipelines in Python and ...Show moreLast updated: 30+ days ago A company is looking for a Lead Data Engineer to architect and scale their data platform and analytics stack.Key Responsibilities Refine and evolve the AWS data platform to enhance analytical cap...Show moreLast updated: 30+ days ago A company is looking for a Staff Data Engineer - Hybrid.Key Responsibilities Design, build, and maintain scalable data models and ETL / ELT pipelines for enterprise analytics and reporting Develop...Show moreLast updated: 30+ days ago 
Data Engineer 5 years' experience with data management, best practices and software development.Experience in Data Warehousing, data integration and data cleansing.Dev...Show moreLast updated: 30+ days ago Join our team as we reimagine and modernize the core of Relativity's data architecture.You'll play a pivotal role in transforming the Document Data Model (DDM)-a foundational component of our platf...Show moreLast updated: 30+ days ago A company is looking for a Senior Data Engineer to lead the development of scalable data processing and enrichment pipelines.
Key Responsibilities Design and implement scalable AI-powered data pro...Show moreLast updated: 30+ days ago A company is looking for a Staff Engineer to create technical strategies for key systems and ensure platform scalability.
Key Responsibilities : Provide architectural leadership across engineering ...Show moreLast updated: 30+ days ago 
At least 2-3 years of experience working with Informatica.Good understanding of data warehouse and ETL concepts.Very strong in SQL programming.
Teradata - SQL, implementing complex stored and hands-...Show moreLast updated: 14 hours ago A company is looking for a Data Engineer III to design, develop, and maintain data pipelines and systems.Key Responsibilities Design, develop, and maintain data pipelines and infrastructure using...Show moreLast updated: 30+ days ago A company is looking for a Full Stack Engineer (Remote - Healthcare Required).Key Responsibilities Design, build, and maintain efficient, reusable, and reliable code across the full stack Develo...Show moreLast updated: 2 days ago A company is looking for a Lead Data / ML Engineer to build scalable, cost-efficient systems for transforming ad data into reliable intelligence.
Key Responsibilities Design, build, and maintain sca...Show moreLast updated: 2 days ago A company is looking for a Data Engineer to design, construct, and maintain scalable data management systems.Key Responsibilities Develop and maintain data architectures, including databases and ...Show moreLast updated: 30+ days ago A company is looking for a Senior Lead Consultant (Databricks Architect).Key Responsibilities Serve as the technical lead for project teams, ensuring high-quality execution and alignment with bes...Show moreLast updated: 30+ days ago A company is looking for a Data Engineer (AI / ML) 2 to support the Generative AI for Forms SBIR project by designing and maintaining data infrastructure for AI applications.Key Responsibilities De...Show moreLast updated: 3 days ago A company is looking for a Cloud Data Operations Engineer (Technology Specialist I).Key Responsibilities Support the planning, design, implementation, and deployment of Data Cloud services on Mic...Show moreLast updated: 2 days ago A company is looking for a Staff Platform Engineer.Key Responsibilities Write Terraform modules and Go libraries for infrastructure management Develop and manage Kubernetes, Docker, and compute ...Show moreLast updated: 30+ days ago
- Promoted
Lead Software Engineer - Data Platform
RelativityColumbus, OH, United StatesFull-time
- Promoted
Data Science Engineer
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
Software Engineer, Molecular Data
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
Senior Elasticsearch Engineer
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
Principal Data Engineer
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
Lead Data Engineer
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
Staff Data Engineer
VirtualVocationsColumbus, Ohio, United StatesFull-time
Data Engineer
Purple DriveCOLUMBUS, Ohio, USAFull-time
- Promoted
Senior Software Engineer - Data Platform
RelativityColumbus, OH, United StatesFull-time
- Promoted
Senior Data Engineer
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
Staff Engineer
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
- New!
Databricks Developer
Diverse LynxColumbus, OH, USFull-time
- Promoted
Data Engineer III
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
Healthcare Full Stack Engineer
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
Lead Data and ML Engineer
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
Data Engineer
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
Senior Databricks Architect
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
Data Engineer (AI / ML)
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
Cloud Data Operations Engineer
VirtualVocationsColumbus, Ohio, United StatesFull-time
- Promoted
Platform Engineer
VirtualVocationsColumbus, Ohio, United StatesFull-time