Skip to content
SpotifySpotifyNew York, NY

Senior Machine Learning Engineer, Personalization, Magenta

Designs and ships production ML systems for conversational AI personalization at Spotify, focusing on user intent interpretation, agentic workflows, context management, and scalable evaluation frameworks. Requires 5+ years experience with LLMs and production ML.

Salary not listed
Remote5+ YOEML Engineering

About the role

What You'll Do

  • Design and ship production-grade machine learning systems powering conversational and agentic AI experiences
  • Build systems that interpret user intent, manage context across multi-turn interactions, and handle ambiguity reliably at scale
  • Develop and evolve agentic workflows including memory, context management, and multi-step tool orchestration
  • Create evaluation frameworks, including LLM-as-judge pipelines, to measure quality and guide iteration
  • Partner closely with product, engineering, and design to deliver seamless, user-facing experiences
  • Balance experimentation with production rigor, ensuring performance, latency, and reliability at Spotify scale
  • Continuously improve agent behavior through tight feedback loops between evaluation and real-world usage

Who You Are

  • 5+ years of experience building and shipping machine learning systems in production environments
  • Experienced with large language models and have worked on real-world applications beyond experimentation; shipped and maintained large scale systems with LLMs
  • Deep understanding of challenges in conversational or agentic systems, such as context handling and multi-step reasoning
  • Know how to evaluate ML systems rigorously and have experience designing metrics or evaluation pipelines
  • Comfortable debugging complex interactions between models, tools, and system constraints like latency
  • Care about building reliable, scalable systems that deliver high-quality user experiences
  • Enjoy working cross-functionally and contributing to a collaborative, inclusive team environment

Skills

Machine LearningLLMsConversational AIAgentic AILlm-As-JudgeContext ManagementMulti-Step ReasoningEvaluation FrameworksProduction Ml Systems

Similar roles

ML Engineering jobs
Scale AI

Senior Machine Learning Engineer, Agent Oversight

Scale AISan Francisco, CA +1

Senior ML Engineer building observability, evaluation frameworks, and improvement loops for production agentic AI systems. Requires 5+ years production ML/LLM experience, strong grounding in agent design or evaluation, and hands-on work taking systems from prototype to scale.

216k – 270k/yr
On-site5+ YOEML Engineering
Scale AI

Senior Software Engineer, Agent Oversight

Scale AISan Francisco, CA +1

Build platform infrastructure for observing, evaluating, and improving production AI agents at scale. Requires 4+ years software engineering experience with ML/LLM systems, strong backend/distributed systems skills, and collaboration with ML engineers.

216k – 270k/yr
On-site5+ YOEML Engineering
Ambient.ai

Senior Software Engineer, AI Infrastructure

Ambient.aiRedwood City, CA

Build and optimize scalable AI infrastructure for real-time inference, evaluation, and continuous improvement of LLMs, LVMs, computer vision, and multimodal models on large-scale video data. Requires 4+ years production ML systems experience, strong Python skills, and expertise in inference optimization and model serving.

168k – 205k/yr
Hybrid4+ YOEML Engineering
SpotOn

Senior Software Engineer

SpotOnChicago, IL +3

Senior AI Engineer building agentic workflows and orchestration layers to automate manual business processes for operations, sales, and customer support teams. Requires 7+ years software engineering experience, production LLM/agent expertise, and strong Python/TypeScript skills.

160k – 190k/yr
Hybrid7+ YOEML Engineering
Snowflake

Senior AI Engineer, Enterprise

SnowflakeMenlo Park, CA

Senior AI Engineer building scalable backend services, distributed systems, and reusable AI platform components (agent frameworks, evaluation pipelines, SDKs) that power Snowflake's internal agentic applications for GTM teams. Requires 5+ years software engineering experience, strong Python/FastAPI skills, and production LLM experience.

156k – 224k/yr
On-site5+ YOEML Engineering