Databricks Engineer - Commercial Insurance - FulltimeAcunor Infotech • Columbus, OH, US

Databricks Engineer - Commercial Insurance - Fulltime

Acunor Infotech • Columbus, OH, US

4 days ago

Job type

Full-time

Job description

ob Title : Data Engineer (Databricks Azure)

Client : One of our Consulting Clients (Global Analytics & Digital Transformation Firm)

Location : Columbus, OH (Remote / Hybrid)

Duration : Full-Time

About the Role

We are seeking a highly skilled Data Engineer with deep expertise in Databricks and Azure Cloud to join a decision analytics and data engineering team within one of our global consulting clients. The role involves building, optimizing, and maintaining large-scale data pipelines that fuel enterprise analytics, reporting, and AI-driven insights-primarily supporting clients in the insurance and financial services domains

Key Responsibilities

Data Pipeline Development & Optimization

Design, build, and enhance ETL / ELT data pipelines using Azure Data Factory Databricks (PySpark, SQL, Python) , and related services.
Develop and manage Delta Live Tables Autoloader , and Unity Catalog within the Databricks ecosystem for structured, incremental data processing.
Implement data ingestion, transformation, and validation frameworks that ensure high performance, scalability, and reliability.
Monitor data pipelines, troubleshoot issues, and ensure optimal system performance and SLA adherence.

Data Modeling & Architecture

Collaborate with business analysts and reporting teams to define logical and physical data models supporting analytical and operational needs.

Implement data warehousing and lakehouse solutions using Azure Data Lake and Delta Lake

Optimize data structures for query performance, cost efficiency, and reusability.

Data Quality, Governance & Automation

Design and implement robust data quality checks and validation mechanisms to maintain integrity across sources and transformations.

Automate repetitive processes using scripts, parameterized pipelines, and reusable frameworks.

Conduct periodic audits and compliance checks aligned with governance policies.

Contribute to metadata management, documentation, and lineage tracking.

Required Skills & Experience

7 12 years of experience in Data Engineering with proven expertise in Databricks and Azure Cloud ecosystems.

Strong hands-on experience in PySpark Python , and SQL for data transformation, validation, and performance tuning.

Solid understanding of Delta Lake architecture ETL / ELT frameworks , and data warehousing principles.

Proficiency with Azure services including Data Factory (ADF) Data Lake (ADLS) , and Databricks Notebooks

Experience with Delta Live Tables Unity Catalog , and Autoloader for batch and streaming data processing.

Strong background in data modeling performance optimization , and automation scripting

Familiarity with Agile methodologies and DevOps-based deployment practices (Git, CI / CD preferred).

Strong analytical, communication, and problem-solving skills to collaborate effectively across diverse teams.

Preferred : Exposure to insurance, healthcare, or financial services data ecosystems.

Nice to Have

Experience in data migration projects (on-prem to cloud or multi-cloud).

Familiarity with Delta Sharing Databricks SQL Warehouses , or MLflow for advanced use cases.

Experience with data cataloging, lineage, or quality frameworks such as Purview Collibra , or Great Expectations

Exposure to BI / reporting tools like Power BI or Tableau for end-to-end integration understanding.

J-18808-Ljbffr

Create a job alert for this search

Commercial Insurance • Columbus, OH, US

Related jobs

Databricks Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Databricks Engineer.Key Responsibilities Design, develop, and deploy cloud data solutions for risk, regulatory, finance, and compliance projects Provide indirect leade...Show more

Last updated: 30+ days ago • Promoted

Distinguished Engineer - Cloud-Native

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Distinguished Engineer - Cloud-Native Identity and Data Platform.Key Responsibilities Architect and evolve the Identity and Data Platform to support a growing SaaS port...Show more

Last updated: 1 day ago • Promoted

Full-Stack Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Full-Stack Engineer : Sponsored Listings (Remote).Key Responsibilities Design, deliver, and maintain a web application in collaboration with other engineers Work with P...Show more

Last updated: 30+ days ago • Promoted

Cloud Data Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Cloud Data Engineer to join the Data Office Team in modernizing enterprise analytics.Key Responsibilities Design and develop data pipelines and ELT processes to integra...Show more

Last updated: 30+ days ago • Promoted

Lead Software Engineer - Data Platform

Relativity • Columbus, OH, United States

Full-time

Join our team as we reimagine and modernize the core of Relativity's data architecture.You'll play a pivotal role in transforming the Document Domain - a foundational component of our platform-into...Show more

Last updated: 30+ days ago • Promoted

ETL Data Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Data Engineer for a contract role.Key Responsibilities Develop Ab Initio jobs and workflows within the existing metadata-driven framework Review requirements and sourc...Show more

Last updated: 3 days ago • Promoted

Azure Cloud Data Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Data Engineer for a fully remote contract opportunity.Key Responsibilities Manage data / cloud generalist responsibilities related to Oracle, DevOps, and Azure Cloud Bui...Show more

Last updated: 2 days ago • Promoted

Data Engineer / DBA

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Data Engineer / DBA to manage and optimize their data systems.Key Responsibilities Design and manage MySQL databases focusing on speed, scalability, and reliability Bu...Show more

Last updated: 30+ days ago • Promoted

Principal Data Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Principal Data Engineer to lead data architecture and platform initiatives.Key Responsibilities Define and implement end-to-end data architecture and establish standard...Show more

Last updated: 30+ days ago • Promoted

AWS Data Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for an AWS Data Engineer to support cloud migration and optimize data queries.Key Responsibilities Review and rewrite existing SQL / Power BI queries for Amazon Redshift Optim...Show more

Last updated: 30+ days ago • Promoted

Data Pipeline Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Staff Data Pipeline Engineer.Key Responsibilities Design, build, and optimize high-performance data pipelines for business and customer-facing systems Implement and ma...Show more

Last updated: 30+ days ago • Promoted

Staff Data Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Staff Data Engineer - Databricks SME (Remote).Key Responsibilities Develop and maintain ETL / ELT pipelines using Databricks notebooks and workflows Optimize Spark jobs ...Show more

Last updated: 30+ days ago • Promoted

Full Stack Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Full Stack Engineer to lead the development of a platform.Key Responsibilities Design, develop, and maintain front-end and back-end systems for the platform Write clea...Show more

Last updated: 30+ days ago • Promoted

Senior Data Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Senior Data Engineer to oversee data integration and migration, develop data warehouses, and mentor junior engineers. Key Responsibilities Developing solutions to improv...Show more

Last updated: 30+ days ago • Promoted

Senior Software Engineer - Data Platform

Relativity • Columbus, OH, United States

Full-time

Join our team as we reimagine and modernize the core of Relativity's data architecture.You'll play a pivotal role in transforming the Document Data Model (DDM)-a foundational component of our platf...Show more

Last updated: 30+ days ago • Promoted

Lead Data Platform Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Lead Data Platform Engineer to architect and build scalable distributed data systems for cloud-based video surveillance and IoT control systems.Key Responsibilities Col...Show more

Last updated: 30+ days ago • Promoted

Sr Databricks Platform Engineer

RXC RxC Acquisition Company • Columbus, OH, United States

Full-time

McKesson is an impact-driven, Fortune 10 company that touches virtually every aspect of healthcare.We are known for delivering insights, products, and services that make quality care more accessibl...Show more

Last updated: 10 days ago • Promoted

Data Solutions Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Data Solutions Engineer to support strategic data initiatives by designing and developing data pipelines and models. Key Responsibilities Architect and build new data pi...Show more

Last updated: 30+ days ago • Promoted

Data Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for a Data Systems Engineer to audit, compare, validate, and assess infrastructure and application data in ServiceNow. Key Responsibilities Design, write, and optimize complex...Show more

Last updated: 30+ days ago • Promoted

IT Data Engineer

VirtualVocations • Columbus, Ohio, United States

Full-time

A company is looking for an IT Data Engineer for a contract role in Portsmouth, NH (Remote).Key Responsibilities Participate in Scrum team activities, including standups and sprints, while suppor...Show more

Last updated: 30+ days ago • Promoted