Talent.com
Senior Big Data Hadoop ML Engineer (GCP)

Senior Big Data Hadoop ML Engineer (GCP)

RackspaceRemote, Remote, United States
30+ days ago
Job type
  • Full-time
Job description

About the Role :

We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. The ideal candidate will have a strong background in developing batch processing systems, with extensive experience in the Apache Hadoop ecosystem (Map Reduce, Oozie, Hive, Pig, HBase, Storm).  This role involves working in Java, and working on Machine Learning pipelines for data collection or batch inference.   This is a remote position, requiring excellent communication skills and the ability to solve complex problems independently and creatively.

Work Location : US-Remote

\n

What you will be doing :

  • Develop scalable and robust code for large scale batch processing systems using Hadoop, Oozie, Pig, Hive, Map Reduce, Spark (Java), Python, Hbase
  • Develop, manage, and maintain batch pipelines supporting Machine Learning workloads
  • Leverage GCP for scalable big data processing and storage solutions
  • Implementing automation / DevOps best practices for CI / CD, IaC, etc.

Requirements :

  • Proficiency in in the Hadoop ecosystem with Map Reduce, Oozie, Hive, Pig, HBase, Storm
  • Strong programming skills with Java, Python, and Spark
  • Knowledge in public cloud services, particularly in GCP.
  • Experienced in Infrastructure and Applied DevOps principles in daily work. Utilize tools for continuous integration and continuous deployment (CI / CD), and Infrastructure as Code (IaC) like Terraform to automate and improve development and release processes.
  • Ability to tackle complex challenges and devise effective solutions. Use critical thinking to approach problems from various angles and propose innovative solutions.
  • Worked effectively in a remote setting, maintaining strong written and verbal communication skills. Collaborate with team members and stakeholders, ensuring clear understanding of technical requirements and project goals.
  • Proven experience in engineering batch processing systems at scale.
  • Hands-on experience in public cloud platforms, particularly GCP. Additional experience with other cloud technologies is advantageous.
  • Must Have :

  • Experience with batch pipelines supporting Machine Learning workloads
  • Strong experience in programming language such as Java
  • Strong experience in the Apache Hadoop ecosystem
  • 10+ years of experience in customer-facing software / technology or consulting
  • 5+ years of experience with “on-premises to cloud” migrations or IT transformations
  • Technical degree : Computer Science, Software Engineering or related
  • Good to Have :

  • Familiarity with Terraform
  • Familiarity with Python
  • 5+ years of experience building, and operating solutions built on GCP
  • \n

    The following information is required by pay transparency legislation in the following states : CA, CO, HI, NY, and WA. This information applies only to individuals working in these states.

  • The anticipated starting pay range for Colorado is : $116,100 - $170,280.
  • The anticipated starting pay range for the states of Hawaii and New York (not including NYC) is : $123,600 - $181,280.
  • The anticipated starting pay range for California, New York City and Washington is : $135,300 - $198,440.
  • Unless already included in the posted pay range and based on eligibility, the role may include variable compensation in the form of bonus, commissions, or other discretionary payments. These discretionary payments are based on company and / or individual performance and may change at any time. Actual compensation is influenced by a wide array of factors including but not limited to skill set, level of experience, licenses and certifications, and specific work location. Information on benefits   offered is here .#LI-MF1 #LI-Remote

    About Rackspace Technology

    We are the multicloud solutions experts. We combine our expertise with the world’s leading technologies — across applications, data and security — to deliver end-to-end solutions. We have a proven record of advising customers based on their business challenges, designing solutions that scale, building and managing those solutions, and optimizing returns into the future. Named a best place to work, year after year according to Fortune, Forbes and Glassdoor, we attract and develop world-class talent. Join us on our mission to embrace technology, empower customers and deliver the future.

    More on Rackspace Technology

    Though we’re all different, Rackers thrive through our connection to a central goal : to be a valued member of a winning team on an inspiring mission. We bring our whole selves to work every day. And we embrace the notion that unique perspectives fuel innovation and enable us to best serve our customers and communities around the globe. We welcome you to apply today and want you to know that we are committed to offering equal employment opportunity without regard to age, color, disability, gender reassignment or identity or expression, genetic information, marital or civil partner status, pregnancy or maternity status, military or veteran status, nationality, ethnic or national origin, race, religion or belief, sexual orientation, or any legally protected characteristic. If you have a disability or special need that requires accommodation, please let us know.

    Create a job alert for this search

    Senior Data Engineer • Remote, Remote, United States

    Related jobs
    • Promoted
    Senior Machine Learning Engineer

    Senior Machine Learning Engineer

    Aily LabsRemote, Remote, United States
    Full-time
    Develop tools and frameworks for model training, tuning, and evaluation, ensuring seamless infrastructure integration.Partner with Data Scientists and Engineers to bring AI solutions and Laboratori...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    SimplepracticeRemote, Remote, United States
    Remote
    Full-time
    SimplePractice is headquartered in Los Angeles, California, but we have team members who work and live across the United States, Dominican Republic, Mexico City, and Ukraine.We are the world’s lead...Show moreLast updated: 30+ days ago
    Senior Data / AI / ML Software Engineer

    Senior Data / AI / ML Software Engineer

    MLabsUS
    Remote
    Full-time
    Quick Apply
    Senior Data / AI / ML Software Engineer.Remote (US Only) or Hybrid (San Francisco Bay Area).K - $300K (Varies by level and location). We are building a cutting-edge.Clinical Data Intelligence platform.W...Show moreLast updated: 9 days ago
    • Promoted
    • New!
    Sr Big Data Engineer - Oozie and Pig (GCP)

    Sr Big Data Engineer - Oozie and Pig (GCP)

    RackspaceRemote, Remote, United States
    Remote
    Full-time
    We are seeking a Senior Big Data Engineer with deep expertise in distributed systems, batch data processing, and large-scale data pipelines. The ideal candidate has strong hands-on experience with O...Show moreLast updated: 1 hour ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    SamsaraRemote, Remote, United States
    Remote
    Full-time
    Samsara’s Revenue Operations AI & Data Team is building the future of how we go to market — with intelligence, personalization, and speed. We’re a high-impact team of builders, scientists, and strat...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Nova TalentUnited States, United States, United States
    Remote
    Full-time
    We are seeking a skilled Data Engineer to join our team and drive our data infrastructure forward.In this role, you will primarily focus on maintaining and enhancing our data warehouse and pipeline...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer, ML Platform

    Senior Data Engineer, ML Platform

    RedditRemote, Remote, United States
    Remote
    Full-time
    Reddit is a community of communities.It’s built on shared interests, passion, and trust and is home to the most open and authentic conversations on the internet. Every day, Reddit users submit, vote...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    PerformlineRemote, Remote, United States
    Remote
    Full-time
    Hybrid (Morristown, NJ) OR Remote (US).PerformLine is a category-leading SaaS company that empowers leaders with end-to-end marketing compliance technology, from automated review of documents to di...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Mission Underwriting ManagersRemote, Remote, United States
    Remote
    Full-time
    The Senior Data Engineer will play a key role in designing, implementing, and optimizing Mission’s data infrastructure as part of our modern data platform initiative. This hands-on engineering role ...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer, US - Contract

    Senior Data Engineer, US - Contract

    Ion GroupRemote, Remote, United States
    Remote
    Full-time
    We are seeking a skilled and experienced Senior Data Engineer to join our innovative team.The ideal candidate will bring deep expertise in modern data engineering technologies, strong collaboration...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    AffirmRemote, Remote, United States
    Full-time
    Affirm is reinventing credit to make it more honest and friendly, giving consumers the flexibility to buy now and pay later without any hidden fees or compounding interest.Affirm is reinventing cre...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Senior Data Engineer

    Senior Data Engineer

    PFFRemote, Remote, United States
    Remote
    Full-time
    Lead the Architectural Redesign : Analyze and re-architect the existing highly-normalized PostgreSQL database, driving the transition to a modern, high-performance analytical data model.Optimize for...Show moreLast updated: 1 hour ago
    • Promoted
    Senior Databricks Data Engineer (Lead Consultant)

    Senior Databricks Data Engineer (Lead Consultant)

    LovelyticsRemote, Remote, United States
    Full-time
    Lovelytics is a Databricks-focused data and AI consulting firm specializing in artificial intelligence, data, and analytics solutions. Since partnering with Databricks in 2019, Lovelytics has experi...Show moreLast updated: 30+ days ago
    • Promoted
    Big Data Engineer

    Big Data Engineer

    Sapphire Software SolutionsRemote, Remote, United States
    Remote
    Full-time
    Big Data Developer who has exp working with,.Spark, Scala, Databricks, SQL, Azure (nice to have).This is a project with our partner In. We already have someone working on this team and are looking t...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Machine Learning Engineer

    Senior Machine Learning Engineer

    CubiCasaRemote, Remote, United States
    Full-time
    We are a production team dedicated to supporting and enhancing production modeling systems.Our focus is on building the most accurate models with the state-of-the-art modeling technology.We build c...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Software Engineer, Data & Machine Learning Infrastructure

    Senior Software Engineer, Data & Machine Learning Infrastructure

    MotionalRemote, Remote, United States
    Remote
    Full-time
    At Motional, data plays a critical role in fueling our ML-centered autonomous driving vehicle.Our robo-taxi fleet collects petabytes of data on the road every day – the Data Mining team is mining &...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Maven ClinicRemote, Remote, United States
    Remote
    Full-time
    Maven is the world's largest virtual clinic for women and families on a mission to make healthcare work for all of us.Maven's award-winning digital programs provide clinical, emotional, and financi...Show moreLast updated: 30+ days ago
    • Promoted
    Forward Deployed Data Engineer

    Forward Deployed Data Engineer

    QventusUnited States, United States, United States
    Full-time
    On this journey for over 12 years, Qventus is leading the transformation of healthcare.We enable hospitals to focus on what matters most : patient care. Our innovative solutions harness the power of ...Show moreLast updated: 30+ days ago