Databricks Engineer Remote Role

Hope youre doing well. We have an open position for a Databricks Engineer . See the details below and let me know your interest. If interested please share a copy of your resume to along with your salary / rate expectations and the best time to reach you.

Role : Databricks Engineer

Client : Maryland State Client

Location : College Park Maryland (Remote Role)

Job ID : 92248

Description :

We are seeking a Databricks Engineer to design build and operate a Data & AI platform with a strong foundation in the Medallion Architecture (raw / bronze curated / silver and mart / gold layers). This platform will orchestrate complex data workflows and scalable ELT pipelines to integrate data from enterprise systems such as PeopleSoft D2L and Salesforce delivering high-quality governed data for machine learning AI / BI and analytics at scale.

Responsibilities :

• Design implement and optimize end-to-end data pipelines on Databricks following the Medallion Architecture principles.
• Build robust and scalable ETL / ELT pipelines using Apache Spark and Delta Lake to transform raw (bronze) data into trusted curated (silver) and analytics-ready (gold) data layers.
• Operationalize Databricks Workflows for orchestration dependency management and pipeline automation.
• Apply schema evolution and data versioning to support agile data development.
• Connect and ingest data from enterprise systems such as PeopleSoft D2L and Salesforce using APIs JDBC or other integration frameworks.
• Implement connectors and ingestion frameworks that accommodate structured semi- structured and unstructured data.
• Design standardized data ingestion processes with automated error handling retries and alerting.
• Develop data quality checks validation rules and anomaly detection mechanisms to ensure data integrity across all layers.
• Integrate monitoring and observability tools (e.g. Databricks metrics Grafana) to track ETL performance latency and failures.
• Implement Unity Catalog or equivalent tools for centralized metadata management data lineage and governance policy enforcement.
• Enforce data security best practices including row-level security encryption at rest / in transit and fine-grained access control via Unity Catalog.
• Design and implement data masking tokenization and anonymization for compliance with privacy regulations (e.g. GDPR FERPA).
• Work with security teams to audit and certify compliance controls.
• Enable data scientists by delivering high-quality feature-rich data sets for model training and inference.
• Support AIOps / MLOps lifecycle workflows using MLflow for experiment tracking model registry and deployment within Databricks.
• Collaborate with AI / ML teams to create reusable feature stores and training pipelines.
• Architect and manage data lakes on Azure Data Lake Storage (ADLS) or Amazon S3 and design ingestion pipelines to feed the bronze layer.
• Build data marts and warehousing solutions using platforms like Databricks.
• Optimize data storage and access patterns for performance and cost-efficiency.
• Maintain technical documentation architecture diagrams data dictionaries and runbooks for all pipelines and components.
• Provide training and enablement sessions to internal stakeholders on the Databricks platform Medallion Architecture and data governance practices.
• Conduct code reviews and promote reusable patterns and frameworks across teams.
• Submit a weekly schedule of hours worked and progress reports outlining completed tasks upcoming plans and blockers.
• Track deliverables against roadmap milestones and communicate risks or dependencies.

Required Qualifications :

Preferred Qualifications :

Key Skills

ASP.NET,Health Education,Fashion Designing,Fiber,Investigation

Employment Type : Full Time

Experience : years

Vacancy : 1