Talent.com

Big data engineer Jobs in USA

Create a job alert for this search

Big data engineer • usa

Last updated: 3 days ago

Big Data Engineer

SGARockville, MD
Full-time

Software Guidance & Assistance, Inc.This role is hybrid ( days/week Onsite in either Rockville or Tysons office).Design, develop, and maintain large-scale data processing pipelines using Big Data t...Show more

Big Data Engineer

SynergisticITCharleston, SC, US
Full-time

Already Know Everything in Tech—But Still Not Hired? Let's Get Real with SynergisticIT.You've mastered Java, Docker, Spring Boot, maybe thrown in some Kubernetes and AWS for good measure.You've rac...Show more

Big Data Engineer

HANMalvern, PA
Full-time

We are seeking an experienced Machine Learning Engineer to join our AI/ML Engineering team.You will be responsible for developing and optimizing complex data pipelines, integrating model pipelines,...Show more

Big Data Engineer Principal

SAICOrlando, FL, US
Full-time

SAIC is seeking an experienced Big Data / Parametric Data Engineer to support an Air Force program out of Orlando, FL.This is an onsite position, with no option for remote work).You must possess an...Show more

Senior Software Engineer (Big Data)

EpsilonWestminster, Colorado, United States
Full-time

Design, develop, and maintain scalable data processing solutions across on-premises and cloud environments using Python and Apache Spark.Optimize and fine-tune Spark jobs for performance, including...Show more

Big Data Engineer

ThrotleRed Bank, NJ, US
Full-time

Please consider joining our technology team at Throtle as a Big Data Developer.In this role, you will be part of the team that is responsible for transforming and maintaining several billion record...Show more

Big Data Hadoop Engineer

Harvey NashCA
Remote
Full-time

We're looking for an outstanding Big Data HadoopEngineer to provide vision, gather requirements and translate client userrequirements into technical architecture and design and implement an integra...Show more

Big Data Software Engineer

GIGATEC EngineeringAnnapolis Junction, MD, US
Full-time
Quick Apply

Perks and Benefits 100% Paid Healthcare 10% 401k in every paycheck 100% Fully Vested!.NOTE – Our positions require a Top Secret clearance , as well as the favorable completion of a polygraph.Applic...Show more

Big Data Engineer

Lorven TechnologiesPiscataway, New Jersey, United States
Full-time

Location: Multi location - USA.Develop and maintain automated data pipelines, ensuring data accuracy and efficient processing using Airflow DAG.Write high-quality, maintainable code in Python to su...Show more

Big Data Engineer

LiniumITChicago, Illinois, United States
Full-time

Our client, a premier investment bank, is looking for a Big Data Engineer to join their growing team in Chicago, IL.Currently our client spends a lot of time trying to answer client's questions by ...Show more

Big Data Engineer

AvanadeMisericórdia
Full-time

Benefit from our flexible work options and career growth opportunities.As an Azure Senior Data Engineer specialized in Microsoft Azure tools, you will support the implementation of projects focused...Show more

Senior Software Engineer (Big Data)

MasterCardO Fallon, IL, US
Full-time +1

Mastercard powers economies and empowers people in 200+ countries and territories worldwide.Together with our customers, we’re helping build a sustainable economy where everyone can prosper.We supp...Show more

Big Data Engineer (GCP)

OmegaHiresPhoenix, AZ, US
Full-time
Quick Apply

Big Data Engineer (GCP) Location: Phoenix, AZ (Onsite/Hybrid) Duration: Job Summary We are seeking an experienced Big Data Engineer with strong expertise in Google Cloud Platform (GCP) to design, b...Show more

Senior Engineer, Big Data

Molina HealthcareUnited States; United States
Full-time

Responsible for all the aspects of architecture, design and implementation of Data Management solution using Big Data platform on Cloudera or Hortonworks and other areas of enterprise application p...Show more

Senior Big Data Engineer

00002 Citibank, N.A.Irving Texas United States
Full-time

Applies in-depth disciplinary knowledge, contributing to the development of new techniques and the improvement of processes and work-flow for the area or function.Integrates subject matter and indu...Show more

Big Data Engineer

HANPA
Full-time

Location: Malvern, PA hybrid (local only).Qualification: (+ minimum experience).AWS Skills - Glue Lambda S KMS Stepfunctions General Skills - Data Transformation Tools like Pandas and Pyspark Cloud...Show more

Sr Big Data Engineer

Headway tekFort Worth, TX, Texas, USA
Full-time

Calibri,sans-serif">Title: Sr Big Data Engineer Show more

Data Engineer Senior (Hadoop/Big Data)

PNC Bank NAPittsburgh , PA
Full-time +1

Leads in developing, supporting and implementing data solutions for multiple applications in order to meet business objectives and user requirements.Leverages technical knowledge and industry exper...Show more

Senior Software Engineer, Big Data

ZINC Zillow, Inc.USA,Remote
Remote
Full-time +1

At Zillow, data is one of our most valuable assets in helping customers unlock life’s next chapter.The Data Platform organization enables teams across Zillow to use that data efficiently, securely,...Show more

People also ask
The top 10 cities are:
Big Data Engineer

Big Data Engineer

SGARockville, MD
20 days ago
Job type
  • Full-time
Job description

Software Guidance & Assistance, Inc., (SGA), is searching for a Big Data Engineer for a Contract assignment with one of our premier Regulatory clients in Rockville, MD or Tysons, VA.

This role is hybrid ( days/week Onsite in either Rockville or Tysons office)

Responsibilities :
  • Design, develop, and maintain large-scale data processing pipelines using Big Data technologies (, Hadoop, Spark, Python, Scala).
  • Implement data ingestion, storage, transformation, and analysis of solutions that are scalable, efficient, and reliable.
  • Stay current with industry trends and emerging Big Data technologies to continuously improve the data architecture
  • Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
  • Optimize and enhance existing data pipelines for performance, scalability, and reliability.
  • Develop automated testing frameworks and implement continuous testing for data quality assurance.
  • Conduct unit, integration, and system testing to ensure the robustness and accuracy of data pipelines.
  • Work with data scientists and analysts to support data-driven decision-making across the organization.
  • Ability to write and maintain automated unit, integration, and end-to-end tests
  • Monitor and troubleshoot data pipelines in production environments to identify and resolve issues.
Requirements :
  • Bachelor's degree in Computer Science, Information Systems or related discipline with at least five () years of related experience, or equivalent training and/or work experience; Master's degree and past Financial Services industry experience preferred.
  • Demonstrated technical expertise in Object Oriented and database technologies/concepts which resulted in deployment of enterprise quality solutions.
  • Past experience with developing enterprise quality solutions in an iterative or Agile environment.
  • Extensive knowledge of industry leading software engineering approaches including Test Automation, Build Automation and Configuration Management frameworks.
  • Strong written and verbal technical communication skills.
  • Demonstrated ability to develop effective working relationships that improved the quality of work products.
  • Should be well organized, thorough, and able to handle competing priorities.
  • Ability to maintain focus and develop proficiency in new skills rapidly.
  • Ability to work in a fast paced environment.
  • Experience with object oriented programming languages such as Java, Scala or Python.
Essential Technical Skills:

Big Data technologies
• Experience with Big data technologies such as Hadoop, Spark, Hive & Trino
• Evaluate understanding of common issues like:
◦ Data skew and strategies to mitigate it.
◦ Working with massive data volumes in PetaBytes.
◦ Troublehsooting job failures due to resource limitations, bad data, scalability challenged.
• Look for real-world debugging and mitigation stories.

AI Skills
• Prompt Engineering: Proficiency in crafting effective prompts for AI coding assistants and analysis tools
• AI Workflow Design: Experience redesigning development processes to leverage AI capabilities
• Data Analysis: Ability to interpret AI-generated insights and translate them into actionable team improvements
• Change Management: Experience leading teams through AI adoption and workflow transformation

SQL Skills (Window Functions, Joins, Complex Queries)
• Assess comfort with SQL window functions, multi-table joins, aggregations.
• Provide examples or ask them to write/optimize SQL queries on the spot.
• Probe how they handle edge cases like NULLs, duplicates, ordering, etc.

Apache Spark (Development, Internals & Tuning)
• Test their understanding of Spark's core architecture — executors, tasks, stages, DAG.
• Focus on Spark performance tuning techniques: partitioning, caching, broadcast joins, etc.
• Ask scenario-based questions on troubleshooting slow running/stuck jobs or resource issues in Spark.
• Explore their experience optimizing Spark jobs for large-scale datasets.

Cloud Technologies
• Check exposure to AWS services like S, EMR, Glue, Lambda, Athena, etc.
• Ask how they've used S with Spark (, dealing with file formats, consistency issues).
• EKS, Serverless knowledge, etc.

Programming - Python or Scala
• Assess ability to write clean, modular, and performant code.
• Look for experience in functional programming concepts (, immutability, higher-order functions).
• Ask about real-world use cases where they wrote scalable data processing code.
• Evaluate understanding of collections, concurrency, and memory management.

Preferred Skills:
• Experience with managing production data pipelines/ETL systems
• Experience with CI/CD
• Experience writing test cases
• AWS certifications