Machine Learning Engineering Manager - LLM Serving & Infrastructure

Machine Learning Engineering Manager - LLM Serving & Infrastructure

The Personalization team makes deciding what to play next on Spotify easier and more enjoyable for every listener. We seek to understand the world of music and podcasts better than anyone else so that we can make great recommendations to every individual and keep the world listening. Hundreds of millions of people use the products we build, including Home and Search, Discover Weekly, and new innovations like AI DJ and AI Playlists. Generative AI is transforming Spotify’s product capabilities and technical architecture. This ML Manager will focus on serving a Unified Recommender model, based on open‑weight LLM and transformer technology. You will collaborate with a diverse team to establish and implement the machine learning plan for the product domain, developing innovative recommendations and agent interactions. As a technology leader, you will manage a team and influence peers, collaborating with internal customers and platform teams to shape the entire Spotify experience.

What You’ll Do

Lead a high‑performing engineering team to develop, build, and deploy a high‑scale, low‑latency LLM serving infrastructure.

Drive the implementation of a unified serving layer to support multiple LLM models and inference types (batch, offline eval flows, and real‑time / streaming).

Lead all aspects of the development of the Model Registry for deploying, versioning, and running LLMs across production environments.

Ensure successful integration with the core Personalization and Recommendation systems to deliver LLM‑powered features.

Define and champion standardized technical interfaces and protocols for efficient model deployment and scaling.

Establish and monitor the serving infrastructure’s performance, cost, and reliability, including load balancing, autoscaling, and failure recovery.

Collaborate closely with data science, machine learning research, and feature teams (Autoplay, Home, Search, etc.) to drive the active adoption of the serving infrastructure.

Scale up the serving architecture to handle hundreds of millions of users and high‑volume inference requests for internal domain‑specific LLMs.

Partner with SRE and ML teams to implement latency and cost optimization techniques such as quantization, pruning, and efficient batching.

Develop observability and monitoring with dashboards, alerting for service health, tracing, A / B test traffic, and latency trends to ensure consistency with defined SLAs.

Contribute to Core LPM Serving, focusing on the technical strategy for deploying and maintaining the core Large Personalization Model (LPM).

Who You Are

5+ years of experience in software or machine learning engineering, with at least 2 years managing an engineering team.

Deep expertise in building, scaling, and governing high‑quality ML systems and datasets, including data schemas, lineage, and validation pipelines.

Strong background in large‑scale, high‑velocity ML / MLOps infrastructure, ideally for personalization, recommendation, or LLMs.

Proven track record of driving complex projects across multiple partners and federated contribution models.

Expertise in designing robust, loosely coupled systems with clean APIs and clear separation of concerns.

Experience integrating evaluation and testing into CI / CD pipelines to enable rapid "fork‑evaluate‑merge" workflows.

Solid understanding of experiment tracking and results visualization platforms.

A pragmatic leader who balances speed with progressive rigor and production fidelity.

Where You’ll Be

This role is based in New York or Boston.

We offer flexibility to work where you work best, with some in‑person meetings but allowing work‑from‑home.

The United States base range for this position is $176,166 – $251,666 plus equity. Benefits include health insurance, six‑month paid parental leave, 401(k) retirement plan, monthly meal allowance, 23 paid days off, 13 paid flexible holidays, and paid sick leave.

Spotify is an equal‑opportunity employer. We welcome you for who you are, no matter where you come from, what you look like, or what’s playing in your headphones.

At Spotify, we are passionate about inclusivity and making our recruitment process accessible to everyone. If you need accommodations during the interview process, please let us know.

Spotify transformed music listening forever when we launched in 2008. Our mission is to unlock the potential of human creativity by giving a million creative artists the opportunity to live off their art and billions of fans the chance to enjoy and be passionate about these creators.

#J-18808-Ljbffr