Talent.com
Senior Staff ML Infra Engineer- Distributed Systems

Senior Staff ML Infra Engineer- Distributed Systems

Advanced Micro Devices, Inc.Santa Clara, CA, United States
14 hours ago
Job type
  • Full-time
Job description

WHAT YOU DO AT AMD CHANGES EVERYTHING

At AMD, our mission is to build great products that accelerate next-generation computing experiences-from AI and data centers, to PCs, gaming and embedded systems. Grounded in a culture of innovation and collaboration, we believe real progress comes from bold ideas, human ingenuity and a shared passion to create something extraordinary. When you join AMD, you'll discover the real differentiator is our culture. We push the limits of innovation to solve the world's most important challenges-striving for execution excellence, while being direct, humble, collaborative, and inclusive of diverse perspectives. Join us as we shape the future of AI and beyond. Together, we advance your career.

Welcome to the Llama team-where curiosity runs wild, and innovation is our natural habitat. We're a tight-knit group of passionate builders, educators, and AI explorers at AMD, united by a shared mission : to push the boundaries of what's possible with generative AI and make cutting-edge knowledge accessible to developers everywhere.

THE ROLE :

AMD is looking for a Senior Staff AI Infra Engineer who is passionate about improving the performance of key applications and benchmarks, with a special focus on AI / ML workloads and GPU-accelerated computing. As an Senior Staff Engineer, you will be a technical leader within a core project and will work at the intersection of hardware and software to optimize performance for next-generation AI applications, including Large Language Models (LLMs) and Agentic AI systems. You will work with the very latest hardware and software technology, providing technical leadership while driving complex technical initiatives.

THE PERSON :

The ideal candidate should be passionate about software engineering and possess strong leadership skills to drive sophisticated issues to resolution. Must demonstrate technical depth and breadth in both traditional computing and emerging AI technologies, with the ability to influence technical direction and mentor other engineers. Able to communicate effectively and work optimally with different teams across AMD.

Key Responsibilities :

  • Lead technical initiatives and provide architectural guidance for AI / ML infrastructure and performance optimization.
  • Optimize and accelerate LLM training and inference on AMD GPUs, improving kernel, communication, and end-to-end system efficiency.
  • Develop and enhance infrastructure supporting LLMs, Agentic AI, and RAG systems.
  • Design, build, and optimize AI workloads on GPU clusters, including large-scale training and inference orchestration, elastic scaling, and workload scheduling across heterogeneous hardware.
  • Debug and resolve complex system-level performance issues across GPU, network, and runtime layers.
  • Drive technical excellence, foster cross-team collaboration, and champion innovation within the organization.

Required Experience :

  • 5+ years of experience in AI / ML infrastructure, distributed systems, or performance-critical software development.
  • Expert-level proficiency in C / C++ and Python.
  • Solid understanding of transformer-based architectures and distributed training frameworks such as Megatron-LM, DeepSpeed, and PyTorch Distributed.
  • Proven experience optimizing LLM training and inference pipelines, including TP / PP / DP / ZeRO parallelism, quantization, and mixed-precision techniques.
  • Hands-on experience designing, building, and scaling training or inference platforms using Kubernetes, Ray, or Kubeflow.
  • Familiarity with GPU architecture and distributed communication libraries (e.g., NCCL, RCCL, MPI), with the ability to analyze and optimize multi-GPU training performance.
  • Experience with profiling and performance-analysis tools for GPU optimization and system-level debugging.
  • Demonstrated technical ownership, strong communication, and problem-solving skills, with a proven record of delivering end-to-end AI / ML infrastructure solutions
  • Preferred Qualifications :

  • In-depth experience with the AMD ROCm ecosystem, including HIP kernel optimization for training and inference.
  • Hands-on experience with model optimization techniques such as quantization, pruning, and distillation for efficient deployment.
  • Knowledge of GPU architecture, memory hierarchy, and compiler-level optimization (e.g., kernel fusion, graph scheduling).
  • Familiarity with Agentic AI systems and autonomous AI workflows, including tool use, reasoning, and multi-agent orchestration for LLM-based applications
  • Education

  • Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, or related field.
  • Master's degree preferred; PhD is a strong plus, especially with publications in distributed systems, AI infrastructure, or GPU computing
  • #LI-TC1

    #LI-HYBRID

    Benefits offered are described : AMD benefits at a glance.

    AMD does not accept unsolicited resumes from headhunters, recruitment agencies, or fee-based recruitment services. AMD and its subsidiaries are equal opportunity, inclusive employers and will consider all applicants without regard to age, ancestry, color, marital status, medical condition, mental or physical disability, national origin, race, religion, political and / or third-party affiliation, sex, pregnancy, sexual orientation, gender identity, military or veteran status, or any other characteristic protected by law. We encourage applications from all qualified candidates and will accommodate applicants' needs under the respective laws throughout all stages of the recruitment and selection process.

    Create a job alert for this search

    Senior Ml Engineer • Santa Clara, CA, United States

    Related jobs
    • Promoted
    Senior Engineer, Data & ML Infrastructure

    Senior Engineer, Data & ML Infrastructure

    ProtingentHillsborough, CA, United States
    Permanent
    Senior Engineer, Data & ML Infrastructure.Protingent Staffing has an exciting Remote Direct Hire Senior Engineer, Data & ML Infrastructure opportunity. Work with ML Engineers and Autonomy Software D...Show moreLast updated: 2 days ago
    • Promoted
    AIML - Staff ML System Engineer, ML Platform Technologies (MLPT)

    AIML - Staff ML System Engineer, ML Platform Technologies (MLPT)

    AppleSanta Clara, CA, United States
    Full-time
    Santa Clara, California, United States Machine Learning and AI.We're searching for strong machine learning engineers to help build next-generation platform for training deep learning models at scal...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Applied AI Engineer – ML for Systems & Infrastructure

    Senior Applied AI Engineer – ML for Systems & Infrastructure

    Databricks Inc.San Francisco, CA, United States
    Full-time
    Senior Applied AI Engineer – ML for Systems & Infrastructure.The Applied AI team at Databricks sits at the forefront of advancing GenAI-powered products. Over the past years, we’ve launched Databric...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Senior Software Engineer, ML Infra

    Senior Software Engineer, ML Infra

    AmbientRedwood City, CA, United States
    Full-time
    AI-powered physical security platform helping the world's leading enterprises reduce risk, improve operational efficiency, and gain critical insights. Fortune 500 organizations, rely on Ambient.Our ...Show moreLast updated: 8 hours ago
    • Promoted
    Senior / Staff Infrastructure Engineer

    Senior / Staff Infrastructure Engineer

    ApiphanySan Francisco, CA, United States
    Full-time
    Apiphany is a pioneering foundational AI company for physical product development.We empower global innovators in automotive, aerospace, medtech, and energy to transform mountains of unstructured t...Show moreLast updated: 17 days ago
    • Promoted
    Senior / Staff Infrastructure Engineer

    Senior / Staff Infrastructure Engineer

    Apiphany CorporationSan Francisco, CA, United States
    Full-time
    Apiphany is a pioneering foundational AI company for physical product development.We empower global innovators in automotive, aerospace, medtech, and energy to transform mountains of unstructured t...Show moreLast updated: 18 days ago
    • Promoted
    Staff ML Platform Engineer – Large Scale Training (LLMOps / MLOps)

    Staff ML Platform Engineer – Large Scale Training (LLMOps / MLOps)

    Socotra, Inc.San Francisco, CA, United States
    Full-time
    Build the Future of Scalable AI at TrueFoundry.ML teams train, deploy, and scale their models.Our LLMOps and MLOps platform empowers organizations to experiment faster, train large-scale models rel...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Senior Systems Engineer : ML Pipelines & GPU-Scale Infra

    Senior Systems Engineer : ML Pipelines & GPU-Scale Infra

    Acceler8 TalentSan Francisco, CA, United States
    Full-time
    A Series A Funded start-up in California is seeking a Systems Engineer to design and optimize systems handling complex ML pipelines. The role involves building scalable infrastructure, developing CI...Show moreLast updated: 8 hours ago
    • Promoted
    Staff ML Platform Engineer, Scalable Infra & Search

    Staff ML Platform Engineer, Scalable Infra & Search

    Apple Inc.San Francisco, CA, United States
    Full-time
    A leading technology company seeks a Staff ML Engineer for their Machine Learning Platform Technologies team in San Francisco. The role involves building scalable infrastructures for machine learnin...Show moreLast updated: 2 days ago
    • Promoted
    Staff ML Engineer

    Staff ML Engineer

    GrindrSan Francisco, CA, United States
    Full-time
    San Francisco or Palo Alto offices (Palo Alto preferred) and will require you to be in the office on Tuesdays and Thursdays. What’s So Interesting About This Role?.At Grindr, we’re at the dawn of an...Show moreLast updated: 30+ days ago
    • Promoted
    Staff / Senior AI / ML Engineer - (Dublin, CA)

    Staff / Senior AI / ML Engineer - (Dublin, CA)

    Articul8Dublin, CA, United States
    Full-time
    At Articul8, we build enterprise-grade Generative AI solutions that help global organizations unlock new value from their data. Our platform is trusted by some of the world's most innovative compani...Show moreLast updated: 30+ days ago
    • Promoted
    Senior ML Infrastructure Engineer - Training Algorithms, SIML

    Senior ML Infrastructure Engineer - Training Algorithms, SIML

    AppleCupertino, CA, United States
    Full-time
    Are you passionate about Generative AI? Are you interested in working on groundbreaking generative modeling technologies to enrich billions of people? We are the Intelligence System Experience (ISE...Show moreLast updated: 1 day ago
    • Promoted
    Staff ML Infrastructure Engineer

    Staff ML Infrastructure Engineer

    Cubiq RecruitmentSunnyvale, CA, US
    Full-time
    Staff / Lead ML Infrastructure Engineer.Salary - Over market average + equity.We are building one of the world's leading generative video and multimodal AI platforms, and we're looking for a senior...Show moreLast updated: 3 days ago
    • Promoted
    Staff ML Engineer : Build & Deploy Scalable ML Pipelines

    Staff ML Engineer : Build & Deploy Scalable ML Pipelines

    Intuit Inc.Mountain View, CA, United States
    Full-time
    A leading company in financial software is seeking a Staff Machine Learning Engineer to join their innovative team.In this role, you'll collaborate with highly skilled AI scientists and engineers t...Show moreLast updated: 1 day ago
    • Promoted
    Senior Staff Engineer, TLM - Mobility Marketplace

    Senior Staff Engineer, TLM - Mobility Marketplace

    UberSan Francisco, CA, United States
    Full-time
    This role sits within Uber's Mobility Marketplace team, focusing on pricing, matching, signals, positioning, and incentives. The team optimizes pricing, incentives, and other levers to balance the m...Show moreLast updated: 3 days ago
    • Promoted
    • New!
    Senior Software Engineer, ML Infrastructure, PrePlan

    Senior Software Engineer, ML Infrastructure, PrePlan

    WaymoSan Francisco, CA, United States
    Full-time
    Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver.Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on buildin...Show moreLast updated: 20 hours ago
    • Promoted
    ML Infrastructure Engineer, Safeguards

    ML Infrastructure Engineer, Safeguards

    AnthropicSan Francisco, CA, United States
    Full-time
    Anthropic’s mission is to create reliable, interpretable, and steerable AI systems.We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group ...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Senior Staff ML Engineer – Hybrid, Build Pipelines & LLMs

    Senior Staff ML Engineer – Hybrid, Build Pipelines & LLMs

    Palo Alto NetworksSanta Clara, CA, United States
    Full-time
    A leading cybersecurity company in Santa Clara is looking for a Sr Staff Data Scientist to build data pipelines and machine learning models. This hybrid role requires strong Python and SQL skills, a...Show moreLast updated: 8 hours ago