Talent.com
Hadoop/Python Engineer
Hadoop/Python EngineerITR Group • Brooklyn Park, MN, United States
Hadoop / Python Engineer

Hadoop / Python Engineer

ITR Group • Brooklyn Park, MN, United States
1 day ago
Job type
  • Full-time
Job description

Contract consulting opportunity for an experienced Machine Learning Engineer with Hadoop / Python experience.

  • Hybrid onsite (approx 2-3 days / week) - Must be local to Minneapolis metro
  • Client work authorization requirements : US Citizen or GC holder
  • W2 hourly only (No C2C)
  • Must have experience with Hadoop, Spark, Python, Orchestration

Seeking a Machine Learning Engineer to support and maintain critical batch jobs responsible for generating high-volume forecasts. These jobs are scheduled using a proprietary in-house scheduler and run R, Python, Bash, and Spark on an in-house YARN-managed cluster.

You will play a key role in enabling smooth day-to-day operations by watching, restarting, and debugging jobs in order to minimize downtime and system reliability. Besides operations, you will also contribute towards code enhancements, performance optimization, and observability instrumentation to improve workflow efficiency and stability.

Responsibilities

  • Manage, debug, and oversee day-to-day batch jobs and Spark-based forecasting pipelines.
  • Diagnose and resolve performance, reliability, and dependency issues in distributed environments.
  • Develop and scale existing roles with Python (PySpark), Bash, and Terraform.
  • Develop and automate observability and monitoring features with OpenTelemetry (OTEL), Kibana, Grafana, and bespoke REST APIs.
  • Collaborate with cross-functional groups (Data Science, DevOps, Infrastructure) to integrate high system reliability.
  • Make code contributions to versioned repositories with GitHub, pull requests, code reviews, and branching models.
  • Identify continuous improvement opportunities in orchestration, monitoring, and job performance.
  • Must Have Skills

  • Apache Hadoop, Apache Hive, Apache Spark, and the Apache Spark ecosystem
  • Big Data technologies and distributed computing concepts
  • PySpark and Python development background
  • Docker for containerized environments
  • Git / GitHub for version control and collaboration
  • Nice to Have Skills

  • Workflow orchestration experience with Airflow or similar
  • Bash scripting for automation and environment setup
  • Experience with Grafana or Kibana for observability and monitoring
  • Knowledge of MLOps principles and tooling
  • Knowledge of OpenTelemetry (OTEL) for metrics and tracing
  • Working experience with R for model run or integration
  • Terraform for infrastructure-as-code management
  • Soft Skills

  • Excellent analytical and troubleshooting skills with high focus on details
  • Excellent communication and collaboration in cross-functional teams
  • Proactive, ownership-focused mindset with focus on reliability and performance optimization
  • This is an opportunity to make a tangible impact on a large-scale forecasting platform, working at the intersection of Machine Learning Operations, Data Engineering, and Infrastructure Automation. You'll collaborate with a technically strong team and contribute to both system stability and innovation.

    ITR Group offers a competitive compensation and benefits package, including medical, dental, and 401(k) for eligible employees. The W2 pay range for this type of role is approximately $70.00-89.00 per billable hour. This range is an estimate and not a guarantee of compensation. The final rate will be determined by factors such as experience, market trends, and specific job assignments. Discover more about how ITR Group connects top talent with leading client opportunities.

    Create a job alert for this search

    Engineer • Brooklyn Park, MN, United States

    Related jobs
    Data Architect

    Data Architect

    Genesis10 • Eagan, MN, US
    Permanent +1
    Genesis10 is currently seeking a Data Architect for a 6-month contract role with our healthcare client.Compensation : $70-$90– per hour, depending on skill and experience level.Summary : The Data Arc...Show more
    Last updated: 5 days ago • Promoted
    Entry Level Data Engineer

    Entry Level Data Engineer

    Genesis10 • Minneapolis, MN, US
    Full-time
    To apply, please submit a formal application through our website : Dev10 Apply Here Please note : Applications submitted through this job posting will not be reviewed. Important : This role requires wi...Show more
    Last updated: 30+ days ago • Promoted
    PowerBI Developer

    PowerBI Developer

    Omni Inclusive • Minneapolis, MN, United States
    Full-time
    A Power BI Developer is responsible for designing, developing, and maintaining business intelligence solutions.They work with various data sources to create interactive and user-friendly dashboards...Show more
    Last updated: 30+ days ago • Promoted
    Avionics System Integration Engineer

    Avionics System Integration Engineer

    Butler Aerospace and Defense • Apple Valley, MN, US
    Temporary
    Avionics System Integration Engineer.The engineer who fills this position must have systems, development, troubleshooting and problem solving skills. The primary software languages used are Visual B...Show more
    Last updated: 30+ days ago • Promoted
    Principal, Data & AI Architect

    Principal, Data & AI Architect

    Entegris • Chaska, MN, United States
    Full-time
    Principal, Data & AI Architect.Here at Entegris, we use advanced science to enable technologies that transform the world, and we are seeking employees who have the drive to continue that mission.We...Show more
    Last updated: 30+ days ago • Promoted
    Hadoop Developer ( W2 Candidtaes )

    Hadoop Developer ( W2 Candidtaes )

    360 IT Professionals • Eden Prairie, MN, United States
    Full-time
    IT Professionals is a Software Development Company based in Fremont, California that offers complete technology services in Mobile development, Web development, Cloud computing and IT staffing.Merg...Show more
    Last updated: 1 day ago • Promoted
    Tableau Developer - Minneapolis, Dallas, Chandler, Charlotte, Phoenix, San Francisco

    Tableau Developer - Minneapolis, Dallas, Chandler, Charlotte, Phoenix, San Francisco

    Staffing the Universe • Minneapolis, MN, United States
    Temporary
    Location : Minneapolis, Dallas, Chandler, Charlotte, Phoenix, San Francisco (Remote till COVID) 24 months contract Salary : DOE $ / hr. On w2 US Citizen and Green Card only No Third Party Requirements 7...Show more
    Last updated: 1 day ago • Promoted
    Informatica MDM Consultant

    Informatica MDM Consultant

    Genesis10 • Eagan, MN, US
    Permanent +1
    Genesis10 is currently seeking an Informatica MDM Developer for a 6+ month contract to hire role with our healthcare client. Compensation : $70-$90 per hour, depending on skill and experience level.J...Show more
    Last updated: 5 days ago • Promoted
    Data Engineer Senior

    Data Engineer Senior

    Associated Bank - Corp • Minneapolis, MN, United States
    Full-time
    At Associated Bank we strive to create an inclusive culture where different perspectives are valued and recognized as strengths critical to our success. If you thrive in an environment where your gr...Show more
    Last updated: 30+ days ago • Promoted
    GCP Data Engineer

    GCP Data Engineer

    TEKsystems • Eagan, MN, United States
    Full-time
    In context of this service, TEKsystems team will work on developing a web application that will allow RPh (Pharmacist) to receive and document member intervention plans and outreach activity that w...Show more
    Last updated: 3 days ago • Promoted
    Oracle Developer IV

    Oracle Developer IV

    Genesis10 • Eagan, MN, US
    Permanent
    Genesis10 is currently seeking an Oracle Developer (ORMB / ERP) with our client in the healthcare industry.This is a 1 year+ contract position. Compensation : $45-$55 / hr W2, depending on skill and expe...Show more
    Last updated: 20 days ago • Promoted
    Data Engineering Consultant - Hybrid

    Data Engineering Consultant - Hybrid

    Securian • Saint Paul, MN, United States
    Full-time
    Securian has an amazing opportunity for someone that is curious, eager to learn, and excited to be part of an innovative team using data and technology to tackle business challenges.As a Senior Dat...Show more
    Last updated: 30+ days ago • Promoted
    Tableau Developer - St. Louis, MO; Morris Plains, NJ; Bloomington, MN

    Tableau Developer - St. Louis, MO; Morris Plains, NJ; Bloomington, MN

    Staffing the Universe • Minneapolis, MN, United States
    Full-time
    Louis, MO; Morris Plains, NJ; Bloomington, MN 6 months+ USC / GC and GC EAD.Work with team to develop / enhance the existing dashboard in Tableau. Skills and Experience Required : .Show more
    Last updated: 1 day ago • Promoted
    Entry Level PowerBi Developer

    Entry Level PowerBi Developer

    SynergisticIT • St Paul, MN, United States
    Full-time
    Since 2010 Synergisticit has helped Jobseekers get employed in the tech Job market by providing candidates the requisite skills, experience and technical competence to outperform at interviews and ...Show more
    Last updated: 30+ days ago • Promoted
    Data Architect

    Data Architect

    Medica • Minnetonka, MN, United States
    Full-time
    Medica is a nonprofit health plan with more than a million members that serves communities in Minnesota, Nebraska, Wisconsin, Missouri, and beyond. We deliver personalized health care experiences an...Show more
    Last updated: 30+ days ago • Promoted
    AI Tools Enablement Principle

    AI Tools Enablement Principle

    Genesis10 • Eagan, MN, US
    Permanent
    Job Description : Genesis10 is currently seeking a AI Tools Enablement Principle with our client in the healthcare industry in their Eagan, MN location. This is a 6 month + contract to hire position....Show more
    Last updated: 20 days ago • Promoted
    Hadoop developer

    Hadoop developer

    krg technology inc • Eden Prairie, MN, United States
    Full-time
    Experience Big Data platform coder.Experience with Oozie, HBASE, HIVE, JAVA, PIG, SPARK.Code Big data program using Sqoop and other Big Data Technology. Work with clients, Architect, System Analysts...Show more
    Last updated: 1 day ago • Promoted
    Senior SAP ABAP Developer

    Senior SAP ABAP Developer

    Patterson Companies • St Paul, MN, United States
    Full-time
    Patterson isn't just a place to work, it's a partner that cares about your success.One of the distinguishing marks of our company is the talented people who embrace the people-first, always advanci...Show more
    Last updated: 30+ days ago • Promoted