Data Engineer - Azure & Databricks Focus

Job Description

Job Description

Description

Due to our continued growth, we are seeking a Data Engineer with deep experience in the Azure ecosystem, particularly Azure Databricks, Delta Lake, and SQL, to support the build-out of our medallion data platform.

In this role, you will design and deliver high-performance, governed data pipelines that integrate data from multiple sources—SQL Server Managed Instance, other Azure-based systems, and third-party platforms—using PySpark, SQL, and Databricks utilities.

You will collaborate with data analytics teams and business stakeholders to ensure the datasets produced enable efficient self-service analytics and reporting. Experience working with AI / ML, or a strong interest in developing a platform that supports machine learning, is highly valued.

Mortgage or financial-services experience is a plus, but not required.

Responsibilities and Duties :

• Design, develop, and optimize data pipelines in Azure Databricks using PySpark and SQL, applying Delta Lake and Unity Catalog best practices.
• Build modular, reusable libraries and utilities within Databricks to accelerate development and standardize workflows.
• Implement Medallion architecture (Bronze, Silver, Gold layers) for scalable, governed data zones.
• Integrate external data sources via REST APIs, SFTP file delivery, and SQL Server Managed Instance, implementing validation, logging, and schema enforcement.
• Utilize parameter-driven jobs and manage compute using Spark clusters and Databricks serverless. Collaborate with data analytics teams and business stakeholders to understand requirements and deliver analytics-ready datasets.
• Monitor and troubleshoot Azure Data Factory (ADF) pipelines (jobs, triggers, activities, data flows) to identify and resolve job failures and data issues.
• Automate deployments and manage code using Azure DevOps for CI / CD, version control, and environment management.
• Contribute to documentation, architectural design, and continuous improvement of data engineering best practices.
• Support the design and readiness of the data platform for AI and machine learning initiatives.

Education and Experience :

Preferred Qualifications :

Key Competencies :

How This Role Demonstrates Our Values :

Benefits

Successful applicants will exemplify strong ethics, integrity, respect for others, accountability for decisions and actions, and good citizenship.

Maintaining a reliable, uninterrupted high speed internet connection is a requirement of hybrid or remote positions.

All job duties and responsibilities must be performed within the guidelines of the Verus Residential Mortgage Employee Handbook and established company policies and procedures. It is the responsibility of each employee to maintain confidentiality of the company, its clients and to follow applicable laws and regulations in the performance of duties.

Verus Mortgage Capital is an equal opportunity employer. All qualified applicants are welcomed to apply and will receive consideration for employment without unlawful discrimination because of a person’s race, religious creed, color, national origin, citizenship status, ancestry, marital status, sex, age, or sexual orientation, or because of a person’s disability or medical condition.