Remote Lead Data Engineer (Databricks/AWS)
Position : Lead Data Engineer (Databricks / AWS)
Duration : 6 Month Contract w / extensions or conversion
Location : Remote ; East Coast!!!
Must Haves :
- 2-3 years exp. as a data engineer
- 3 years of RECENT experience with Data Bricks
- Must be their most relevant experience!!
- Experience with Cloud ( Azure, GCP, or AWS)
- Experience with SQL they will be writing queries
- Lead experience Okay with not just being heads down
- Python / pyspark experience
Plusses :
- AWS Certifications
- Data Governance experience
- Bach Degree
Day to Day :
Insight Global is seeking a Lead Data Engineer to sit remote (EAST COAST) for a large retail client headquartered in the greater Pittsburgh area! Our client recently kicked off a major project to build their own home-grown eCommerce platform and applications that will be used internally and then deployed out into their 600+ retail locations.
In this role, the Data Engineer will be a member of the RISC team and will be a design, develop, and maintain scalable and efficient data pipelines for extracting, transforming, and loading (ETL) data from various sources into our data warehouse leveraging AWS, Databricks, Python and SQL technologies.
Additional responsibilities will include :
- Collaborate with cross-functional teams to understand data requirements and implement solutions that meet business needs.
- Optimize and troubleshoot existing data pipelines for performance and reliability.
- Implement data quality and validation processes to ensure accuracy and consistency of data.
- Stay updated on industry trends and best practices in data engineering to continuously improve our data infrastructure.
- Work closely with data scientists, analysts, and other stakeholders to understand data use cases and provide the necessary infrastructure and support.
- Assemble large, complex sets of data that meet non-functional and functional business requirements.
- Identify, design and implement internal process improvements including re-design infrastructure for greater scalability, optimize data delivery, and automating manual processes.
Compensation : $70.00-80.00 / HR
Lead Data / AI Engineer
Under the general direction of the Data Services Director, the Lead Data / AI Engineer will drive the development and implementation of innovative generative AI solutions to enhance audit efficiency and improve data analysis and risk assessment. AWS developing data ingestion, data processing, genera...
Data Engineer, Finance Data Team (REMOTE)
Experience developing new and enhancing existing data processing (Data Ingest, Data Transformation, Data Store, Data Management, Data Quality) components. Data processing/data transformation using ETL/ELT tools such as DBT (Data Build Tool), or Databricks. Our Engineer II is a key member of the engi...
Lead Data / AI Engineer
Under the general direction of the Data Services Director, the Lead Data / AI Engineer will drive the development and implementation of innovative generative AI solutions to enhance audit efficiency and improve data analysis and risk assessment. AWS developing data ingestion, data processing, genera...
Data Engineer - Remote
Working grasp of data modelling. General understanding of batch and streaming data patterns and technologies. Good grasp of TDD and data validation/QA. Exposure to Big Data scenarios and distributed computing tools/frameworks, experience in our tool set a plus. ...
AWS Data Engineer
Proficient in one language (preferably Python) and AWS services. Experience in extracting data - Glue jobs, Redshift and Dynamo DB. Experience in integrating systems with AWS components Lambda, SNS, SQS, S3, Secrets Manager. ...
AWS Data Engineer
Proficient in one language (preferably Python) and AWS services. Experience in extracting data - Glue jobs, Redshift and Dynamo DB. Experience in integrating systems with AWS components Lambda, SNS, SQS, S3, Secrets Manager. ...
Senior Software Engineer - Distributed Data Systems
As a software engineer on the Runtime team at Databricks, you will be building the next generation distributed data storage and processing systems that can outperform specialized SQL query engines in relational query performance, yet provide the expressiveness and programming abstractions to support...
Senior Data Engineer, AWS Energy Team
Our Data Engineers develop large data models for clients in energy, renewable resources, sustainability, and water sectors, enabling them to assess their portfolios. AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. We support all AWS da...
Senior Data Engineer - Innovate with a Startup in AWS, Global Compliance Intelligence
In this role, you will have ownership of an end-to-end development of data engineering solutions to complex questions and you’ll play an integral role in strategic decision-making with our teams of software developers, business intelligence engineers, and data engineers. Come join our new teams in A...
Staff Engineer - Data Management Platform (REMOTE)
Understanding of metadata management, data Ingestion, data management, data quality, and data lineage services and technologies. Experience in data management to include metadata management, data lineage and data quality experience. You will have the opportunity to design and deliver the centralized...