Talent.com
Senior Data Engineer (Python / PySpark / AWS)

Senior Data Engineer (Python / PySpark / AWS)

Infinitive IncAshburn, VA, US
30+ days ago
Job type
  • Full-time
Job description

Job Description

Job Description

  • Candidates must be local to the Washington D.C. metro area.

About Infinitive :

Infinitive is a data and AI consultancy that enables its clients to modernize, monetize and operationalize their data to create lasting and substantial value. . We possess deep industry and technology expertise to drive and sustain adoption of new capabilities. We match our people and personalities to our clients' culture while bringing the right mix of talent and skills to enable high return on investment.

Infinitive has been named “Best Small Firms to Work For” by Consulting Magazine 7 times most recently in 2024. Infinitive has also been named a Washington Post “Top Workplace”, Washington Business Journal “Best Places to Work”, and Virginia Business “Best Places to Work.”

We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. As a Data Engineer, you will play a crucial role in designing, developing, and maintaining our clients data infrastructure. Your expertise in Python, PySpark, ETL processes, CI / CD (Jenkins or GitHub), and experience with both streaming and batch workflows will be essential in ensuring the efficient flow and processing of data to support our clients.

Responsibilities :

Data Architecture and Design :

  • Collaborate with cross-functional teams to understand data requirements and design robust data architecture solutions.
  • Develop data models and schema designs to optimize data storage and retrieval.
  • ETL Development :

  • Implement ETL processes to extract, transform, and load data from various sources.
  • Ensure data quality, integrity, and consistency throughout the ETL pipeline.
  • Python and PySpark Development :

  • Utilize your expertise in Python and PySpark to develop efficient data processing and analysis scripts.
  • Optimize code for performance and scalability, keeping up-to-date with the latest industry best practices.
  • Data Integration :

  • Integrate data from different systems and sources to provide a unified view for analytical purposes.
  • Collaborate with data scientists and analysts to implement solutions that meet their data integration needs.
  • Streaming and Batch Workflows :

  • Design and implement streaming workflows using PySpark Streaming or other relevant technologies.
  • Develop batch processing workflows for large-scale data processing and analysis.
  • CI / CD Implementation :

  • Implement and maintain continuous integration and continuous deployment (CI / CD) pipelines using Jenkins or GitHub Actions.
  • Automate testing, code deployment, and monitoring processes to ensure the reliability of data pipelines.
  • Qualifications :

  • Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
  • 7+ years of proven experience as a Data Engineer or similar role.
  • Strong programming skills in Python and expertise in PySpark for both batch and streaming data processing.
  • Hands-on experience with ETL tools and processes.
  • Familiarity with CI / CD tools such as Jenkins or GitHub Actions.
  • Solid understanding of data modeling, database design, and data warehousing concepts.
  • Excellent problem-solving and analytical skills.
  • Strong communication and collaboration skills.
  • Preferred Skills :

  • Knowledge of cloud platforms such as AWS, Azure, or Google Cloud.
  • Experience with version control systems (e.g., Git).
  • Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes).
  • Understanding of data security and privacy best practices.
  • Applicants for employment in the U.S. must possess work authorization which does not require sponsorship by the employer for a visa. Infinitive is an Equal Opportunity Employer.

    Powered by JazzHR

    B6XnkdgurR

    Create a job alert for this search

    Senior Data Engineer • Ashburn, VA, US

    Related jobs
    • Promoted
    • New!
    Senior Data Engineer (Python, Spark, Databricks, AWS)

    Senior Data Engineer (Python, Spark, Databricks, AWS)

    Capital OneWashington, D.C., US
    Full-time +1
    Senior Data Engineer (Python, Spark, Databricks, AWS) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inc...Show moreLast updated: 12 hours ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Standvast FulfillmentColumbia, MD, United States
    Full-time
    Standvast is a modern, tech-driven third-party logistics provider (3PL) that engineers solutions across the entire supply chain, enabling our customers to leverage a national fulfillment network.We...Show moreLast updated: 3 days ago
    • Promoted
    Senior Lead Data Engineer (Python, AWS, SQL, AI, ML)

    Senior Lead Data Engineer (Python, AWS, SQL, AI, ML)

    Capital OneWashington, D.C., US
    Full-time +1
    Senior Lead Data Engineer (Python, AWS, SQL, AI, ML) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, incl...Show moreLast updated: 1 day ago
    • Promoted
    Senior Manager, Data Engineer (AWS)

    Senior Manager, Data Engineer (AWS)

    Capital OneWashington, DC, US
    Full-time +1
    Senior Manager, Data Engineer (AWS).Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative,.At Capital One, you'l...Show moreLast updated: 30+ days ago
    • Promoted
    Lead Platform Engineer (Python, AWS)

    Lead Platform Engineer (Python, AWS)

    Capital OneWashington, DC, US
    Full-time +1
    Lead Platform Engineer (Python, AWS).Do you love building and pioneering in the technology space? Do you enjoy solving complex technical problems in a fast-paced, collaborative, inclusive, and iter...Show moreLast updated: 1 day ago
    • Promoted
    Senior Lead Data Engineer (Python, SQL, AWS)

    Senior Lead Data Engineer (Python, SQL, AWS)

    Capital OneWashington, DC, US
    Full-time +1
    Senior Lead Data Engineer (Python, SQL, AWS).Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative,.At Capital O...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Software Engineer, Back End (Python, SQL, AWS)

    Senior Software Engineer, Back End (Python, SQL, AWS)

    Capital OneWashington, D.C., US
    Full-time +1
    Senior Software Engineer, Back End (Python, SQL, AWS) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inc...Show moreLast updated: 1 day ago
    • Promoted
    • New!
    Senior Lead Software Engineer, Full Stack (Python, Java, AWS)

    Senior Lead Software Engineer, Full Stack (Python, Java, AWS)

    Capital OneWashington, D.C., US
    Full-time +1
    Senior Lead Software Engineer, Full Stack (Python, Java, AWS) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborat...Show moreLast updated: 12 hours ago
    • Promoted
    Senior Data Engineer (Python, Scala, AWS Cloud)

    Senior Data Engineer (Python, Scala, AWS Cloud)

    Capital OneWashington, D.C., US
    Full-time +1
    Senior Data Engineer (Python, Scala, AWS Cloud) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive...Show moreLast updated: 1 day ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Blend360Columbia, MD, United States
    Permanent
    BLEND360 is an acclaimed, forward-thinking Data, Digital Marketing, & AI Solutions Company, dedicated to fueling remarkable outcomes for our Fortune 500 clients. Our trajectory is one of continuous ...Show moreLast updated: 3 days ago
    • Promoted
    Senior Software Engineer, Back End (Python, Spark, AWS) (Bank Tech)

    Senior Software Engineer, Back End (Python, Spark, AWS) (Bank Tech)

    Capital OneWashington, D.C., US
    Full-time +1
    Senior Software Engineer, Back End (Python, Spark, AWS) (Bank Tech) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, coll...Show moreLast updated: 1 day ago
    • Promoted
    Senior Software Engineer (Java, Python, Golang, AWS)

    Senior Software Engineer (Java, Python, Golang, AWS)

    Capital OneWashington, DC, United States
    Full-time +1
    Senior Software Engineer (Java, Python, Golang, AWS) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, incl...Show moreLast updated: 1 day ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Ad Hoc LLCColumbia, MD, United States
    Full-time
    Ad Hoc is a technology company that empowers organizations to deliver scalable, impactful digital services.Using modern, agile methods, our team creates products that meet people's needs and transf...Show moreLast updated: 3 days ago
    • Promoted
    Senior Data Engineer (Python, Spark, AWS)

    Senior Data Engineer (Python, Spark, AWS)

    Capital OneWashington, D.C., US
    Full-time +1
    Senior Data Engineer (Python, Spark, AWS) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and ...Show moreLast updated: 1 day ago
    • Promoted
    • New!
    Senior Data Engineer (Python, AWS)

    Senior Data Engineer (Python, AWS)

    Capital OneWashington, D.C., US
    Full-time +1
    Senior Data Engineer (Python, AWS) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterati...Show moreLast updated: 12 hours ago
    • Promoted
    Lead Data Engineer (Python / Spark / AWS)

    Lead Data Engineer (Python / Spark / AWS)

    Capital OneWashington, D.C., US
    Full-time +1
    Lead Data Engineer (Python / Spark / AWS) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iter...Show moreLast updated: 1 day ago
    • Promoted
    Senior Data Engineer (Python, Scala, AWS)

    Senior Data Engineer (Python, Scala, AWS)

    Capital OneWashington, DC, United States
    Full-time +1
    Senior Data Engineer (Python, Scala, AWS) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and ...Show moreLast updated: 1 day ago
    • Promoted
    Senior Data Engineer (Azure / Dynamics 365)

    Senior Data Engineer (Azure / Dynamics 365)

    AHU Technologies IncWashington, DC, US
    Full-time
    Client is look for a resource to provide high-level data engineering administration expertise to support Dynamics 365 / Azure data services, including Dataverse management, Azure Data Factory managem...Show moreLast updated: 1 day ago
    • Promoted
    Senior Lead Software Engineer, Full Stack (Java, Python, AWS)

    Senior Lead Software Engineer, Full Stack (Java, Python, AWS)

    Capital OneWashington, D.C., US
    Full-time +1
    Senior Lead Software Engineer, Full Stack (Java, Python, AWS) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborat...Show moreLast updated: 1 day ago
    • Promoted
    • New!
    Senior Software Engineer (Python, Golang, AWS)

    Senior Software Engineer (Python, Golang, AWS)

    Capital OneWashington, D.C., US
    Full-time +1
    Senior Software Engineer (Python, Golang, AWS) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive,...Show moreLast updated: 12 hours ago