Machine Learning Engineering Manager - LLM Serving & Infrastructure

Spotify's Personalization team is seeking an ML Engineering Manager to lead a team building high-scale LLM serving infrastructure. The role involves developing and deploying a unified serving layer for multiple LLM models, managing model registries, and integrating with recommendation systems. The position is based in New York or Boston with flexible work arrangements.

• Lead a high-performing engineering team to develop, build, and deploy a high-scale, low-latency LLM Serving Infrastructure.
• Drive the implementation of a unified serving layer to support multiple LLM models and inference types.
• Lead all aspects of the development of the Model Registry for deploying, versioning, and running LLMs.
• Ensure successful integration with core Personalization and Recommendation systems.
• Define and champion standardized technical interfaces and protocols for efficient model deployment and scaling.
• Establish and monitor serving infrastructure performance, cost, and reliability.
• Collaborate with data science, ML research, and feature teams to drive adoption.
• Scale serving architecture to handle hundreds of millions of users.
• Drive latency and cost optimization via quantization, pruning, and efficient batching.
• Develop observability and monitoring dashboards.
• Contribute to core Large Personalization Model (LPM) serving strategy.