Lead Azure Data EngineerDiligent Tec Inc • Atlanta, GA, Georgia, USA

Lead Azure Data Engineer

Diligent Tec Inc • Atlanta, GA, Georgia, USA

11 days ago

Job type

Full-time

Job description

Title : Lead Data Engineer

Location : Atlanta, GA (Onsite)

Duration : Long Term Contract

Description :

An Lead Azure Data Engineer is responsible for designing, implementing, and maintaining the data infrastructure within an organization. They collaborate with both business and IT teams to understand stakeholders’ needs and unlock the full potential of data. They create conceptual and logical data models, analyze structural requirements, and ensure efficient database solutions.

Must Have Skills :

SAP Data Experience
Proficiency in Databricks and Azure Cloud, Databricks Asset Bundles, Hoslistic vision on the Data Strategy.
Proficiency in Data Streaming and Data Modeling
Experience in architecting at least two large-scale big data projects
Strong understanding of data scaling and its complexities
Data Archiving and Purging mechanisms.

Job Responsibilities :

Act as subject matter expert providing best-practice guidance on data lake and ETL architecture frameworks suitable for handling big data for unstructured and structured information.

Drive business and Service Layer development with the customer by finding new opportunities based on expanding existing solutions and creating new ones.

Provide hands-on subject matter expertise to build and implement Azure-based Big Data solutions.

Research, evaluate, architect, and deploy new tools, frameworks, and patterns to build sustainable Big Data platforms for our clients.

Facilitate and / or conduct requirements workshops.

Responsible for collaborating on the prioritization of technical requirements.

Collaborates with peer teams and vendors on the solution and delivery.

Has overall accountability for project delivery.

Works collaboratively with the Product Management, Data Management, and other Architects to deliver for the cloud data platform and Data as a Service.

Consults with clients to assess current problem states, define desired future states, define solution architecture, and make solutions recommendations.

Design and implement scalable and reusable data models to support analytical and operational use cases, ensuring compatibility with various business applications.

Optimize data models for performance and scalability, addressing key challenges in data latency, volume, and complexity in a big data environment.

Ensure data quality and consistency by establishing robust data validation and governance mechanisms during the modeling process.

Collaborate with data scientists, engineers, and business stakeholders to translate business requirements into effective and efficient data models.

Job Requirements :

Degree in computer science or equivalent preferred

Demonstrable experience in architecture, design, implementation, and / or support of highly distributed applications with Azure cloud and Databricks.

8+ Years of Hands-on experience with data modelling, database design, data mining, and segmentation techniques.

Working knowledge and experience with "Cloud Architectures" (e.g., SaaS, PaaS, IaaS) and the ability to address the unique security considerations of secure Cloud computing

Should have architected solutions for Cloud environments such as Microsoft Azure and / or GCP

Experience with debugging and performance tuning in distributed environments

Strong analytical skills with the ability to collect, organize, analyse, and broadcast significant amounts of information with attention to detail and accuracy

Experience dealing with structured, unstructured data.

Must have Python, PySpark experience.

Experience in ML or / and graph analysis is a plus

Create a job alert for this search

Azure Data Engineer • Atlanta, GA, Georgia, USA

Related jobs

Data Engineer / Analytics Consultant

MCKEE NIX AND ASSOCIATES, INC • Tucker, GA, us

Full-time

Quick Apply

As a Data Engineer / Analytics Consultant at McKee-Nix & Associates, you will be responsible for building and maintaining data infrastructure, pipelines, and analytics solutions that support bu...Show more

Last updated: 5 days ago

Data & BI Senior Data Engineer

Axiom Software Solutions Limited • Atlanta, GA, US

Full-time

Quick Apply

We are seeking a highly skilled and experienced Senior Data Engineer to join our team.The ideal candidate will have a strong background in data engineering, with a specialization in Matillion, SSIS...Show more

Last updated: 30+ days ago

Field Service Manager

Step Up Recruiting • Fayetteville, GA, US

Full-time

Quick Apply

Candidates must be great at leading people.This is a leadership role requiring them to inspire the team and get them to buy into all company goals and standards while feeling appreciated, worthy, a...Show more

Last updated: 30+ days ago • Promoted

Business Insurance Analytics Associate

Marsh LLC • Fayetteville, GA, United States

Full-time

Award-winning, inclusive, Top Workplace culture doesn’t happen overnight.It’s a result of hard work by extraordinary people. The industry’s brightest talent drive our efforts to deliver purposeful w...Show more

Last updated: 19 days ago • Promoted

DCT Break Fix Support - ATL (7 roles)

Apex Systems • Fayetteville, GA, United States

Full-time

Prepares, stages, sets up, and performs startups and shutdowns (e.Performs post-execution quality checks and verifies that grounding, staging, labeling, and cabling are set up properly according to...Show more

Last updated: 15 days ago • Promoted

2nd Shift Bilingual Team Lead

Openwork • Griffin, Georgia, United States

Full-time

Quick Apply

Openwork is currently engaged in a search to find a 2nd shift Bilingual Team Lead in Griffin, GA.In this contract to hire role, you will be responsible for leading a team of assemblers on 2nd shift...Show more

Last updated: 30+ days ago

Senior BI Data Engineer - Oracle EBS

Newsearch Horizons • Atlanta, GA, United States

Full-time

As a Senior BI Technical Data Analyst, you will be responsible for maintaining, designing, developing, optimizing the ETL and data model architecture to support the end user dashboard and reporting...Show more

Last updated: 5 days ago • Promoted

Shift Leader

Panda Restaurant Group • Fayetteville, GA, US

Full-time

Our Panda Shift Leader associates are important leaders of our team and are responsible for bringing Panda’s mission alive in our restaurants by supporting management in creating food with passion,...Show more

Last updated: 30+ days ago

Technical Director - Aerosol Manufacturing

ARI Packaging • Orchard Hill, GA, US

Full-time

Quick Apply

Technical Director – Aerosol Manufacturing Orchard Hill, GA Are you looking to join a company where you're more than just a number and where you and your contributions are truly valued?.At AR...Show more

Last updated: 30+ days ago

Senior / Lead Data Engineer

Cloudious LLC • Alpharetta, GA, United States

Full-time

Quick Apply

Alpharetta, GA is a must 8 openings Visa dependent candidate is fine Salary flexi...Show more

Last updated: 4 days ago

Senior Data Engineer

Plum Inc • Atlanta, GA, US

Remote

Full-time

Quick Apply

PLUM is a fintech company empowering financial institutions to grow their business through a cutting-edge suite of AI-driven software, purpose-built for lenders and their partners across the financ...Show more

Last updated: 30+ days ago

AI / Data Engineer

WickedFile • Alpharetta, Georgia, United States

Full-time

Quick Apply

AI / Data Engineer – Core Role at WickedFile.Automotive SaaS, AI-Driven Analytics.WickedFile is on a mission to redefine how automotive repair shops understand and act on their financial and operatio...Show more

Last updated: 30+ days ago

AWS Data Engineer

TechniPros • Atlanta, GA, United States

Full-time

Quick Apply

Job Title : Senior Data Engineer / Lead Data Engineer Location : Atlanta, Georgia Duration : contract Department : Data Engineering / Enterprise Data Mana...Show more

Last updated: 4 days ago

Data Center Technician

TEKsystems • Fayetteville, GA, United States

Full-time

Will work 100% onsite in Fayetteville GA •.Seeking Level 2, Level 3, Level 4, and Level 5 / Lead Data Center Technicians •. Team will be handling troubleshooting of large GPU deployments inside of a h...Show more

Last updated: 30+ days ago • Promoted

Data Engineer

Wisemen Multimedia, LLC • Chamblee, Georgia, United States

Full-time

Quick Apply

We are hiring an experienced Data Engineer to architect, build, and optimize scalable data pipelines and analytics solutions. The ideal candidate must have hands-on experience with data integration ...Show more

Last updated: 30+ days ago

Senior / Lead Data Engineer (Azure Databricks) - Fulltime Role

Saransh Inc • Alpharetta, GA, United States

Full-time

Quick Apply

Role : Senior / Lead Data Engineer (Azure Databricks) Location : Alpharetta, GA (Onsite) Show more

Last updated: 4 days ago

Senior Data Engineer

Deploy • Atlanta, Georgia, United States

Full-time

Quick Apply

DEPLOY's client is building a next-generation business management platform from the ground up and were looking for a hands-on Senior Data Engineer to deliver and scale our data strategy.As we grow ...Show more

Last updated: 30+ days ago

Technical Specialist-Data Engg

Birlasoft • Alpharetta, GA, United States

Full-time

AWS Relational Database Service.An AWS RDS job description typically involves managing and optimizing relational databases within the AWS cloud using Amazon's RDS service.This includes tasks like p...Show more

Last updated: 8 days ago • Promoted