Azure Data EngineerVivSoft Technologies • Washington, DC, US

Azure Data Engineer

VivSoft Technologies • Washington, DC, US

2 days ago

Job type

Full-time

Quick Apply

Job description

About the Company VivSoft is an emerging technology company specializing in leveraging modern technologies to solve our clients’ most complex mission challenges.

We focus on Cloud, Enterprise DevSecOps, Artificial Intelligence (AI), and Digital Customer Experience to drive mission-enabling digital transformation.

Our passion lies in building mission-focused, open, and scalable solutions.

We are a diverse team of strategists, engineers, designers, and innovators with deep experience in developing high-performance software and AI factory accelerators through automation and agile practices.

Position Overview :

We are seeking a highly skilled Azure Databricks Engineer to design, build, and optimize large-scale data pipelines and lakehouse solutions within the Azure ecosystem.
The ideal candidate will have deep technical expertise in Apache Spark, Delta Lake, and Azure Data Services, along with a strong understanding of data modeling, ETL / ELT processes, and performance optimization.

Key Responsibilities :

Design, build, and maintain ETL / ELT pipelines using Azure Databricks, Delta Lake, and Apache Spark for structured and unstructured data.

Implement data ingestion and transformation processes integrating multiple internal and external data sources.

Architect and optimize data lakehouse solutions, ensuring efficient data storage, retrieval, and partitioning strategies.

Develop and manage batch and real-time streaming data pipelines using Structured Streaming and other Azure services.

Configure and manage Databricks workspaces, clusters, and compute resources for optimal performance and cost efficiency.

Implement data security, access control, and governance measures within the Azure environment.

Collaborate with data scientists to deploy and operationalize machine learning models.

Utilize MLflow for model tracking, versioning, and automated deployment pipelines.

Monitor pipeline performance, troubleshoot issues, and optimize Spark jobs for improved efficiency and reduced processing costs.

Implement data quality checks and ensure reliability across all data processing workflows.

Work closely with stakeholders to translate business needs into scalable data solutions.

Demonstrate strong knowledge of ETL / ELT frameworks, data modeling, and big data technologies.

Hands-on experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and Azure Synapse Analytics.

Proficient in Apache Spark, Delta Lake, Python, SQL, and Scala.

Minimum 5+ years of experience as a Data Engineer, with 3+ years specifically in Azure Databricks and related Azure services.

Skills Required :

Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or a related field.

Strong understanding of data architecture, performance tuning, and cost optimization in Azure.

Excellent analytical, problem-solving, and communication skills.

Experience working in Agile environments and implementing data governance frameworks.

Preferred certifications include Microsoft Certified :

Azure Data Engineer Associate (DP-203) or Databricks Certified Data Engineer.

Passion for innovation, continuous improvement, and driving efficiency in large-scale data environments.

Opportunity to work with cutting-edge Azure data and AI technologies within a collaborative, high-impact environment.

Create a job alert for this search

Azure Data Engineer • Washington, DC, US

Related jobs

Surescripts • Arlington, VA, United States

Full-time

We deliver insights at critical points of care for better decisions - from streamlining prior authorizations to delivering comprehensive medication histories to facilitating messages between provid...Show more

Last updated: 17 days ago • Promoted

ManTech • Alexandria, VA, United States

Full-time

MANTECH seeks a motivated, career and customer-oriented.As part of the position, you will act as a Software Engineer designing and implementing services and components for AI applications.Responsib...Show more

Last updated: 30+ days ago • Promoted

Lead Azure Cloud Engineer

Motion Recruitment • Reston, VA, US

Full-time

As a Lead Azure Cloud Engineer, you will serve as a key contributor within a dynamic, cross-functional engineering team, driving the design, implementation, and optimization of our Azure cloud infr...Show more

Last updated: 15 days ago • Promoted

Software Data Engineer

ManTech • Springfield, VA, US

Full-time

Responsibilities include, but are not limited to : .Provide expert guidance on Enterprise Asset Management; Identify, recommend, and implement agreed upon leading industry practices, process methodol...Show more

Last updated: 9 days ago • Promoted

Cloud Engineer

Viasat • Germantown, MD, United States

Full-time

At Viasat, we're on a mission to deliver connections with the capacity to change the world.For more than 35 years, Viasat has helped shape how consumers, businesses, governments and militaries arou...Show more

Last updated: 4 days ago • Promoted

Sr. Data Platform Engineer

Real Time Medical Systems • Linthicum Heights, MD, US

Full-time

Quick Apply

Data Platform Engineer This is a fully remote role.Job purpose summary Real Time’s suite of Interventional Analytics solutions enables post-acute facilities to seamlessly collaborate with the...Show more

Last updated: 9 days ago

Data Engineer, Public Sector (security clearance required)

Scale AI, Inc. • Washington, DC, United States

Full-time

As a Data Engineer for the Public Sector business unit, you will build Scale's analytical and business-intelligence infrastructure. Scale's customers process millions of tasks through our APIs, and ...Show more

Last updated: 30+ days ago • Promoted

Lead Data Engineer (Enterprise Platforms Technology)

Capital One • McLean, VA, United States

Part-time

Data Engineers who are passionate about marrying data with emerging technologies.As a Capital One Lead Data Engineer, you’ll have the opportunity to be on the forefront of driving a major transform...Show more

Last updated: 15 days ago • Promoted