Designs and ships production ML systems for conversational AI personalization at Spotify, focusing on user intent interpretation, agentic workflows, context management, and scalable evaluation frameworks. Requires 5+ years experience with LLMs and production ML.
Salary not listed
Remote5+ YOEML Engineering
About the role
What You'll Do
Design and ship production-grade machine learning systems powering conversational and agentic AI experiences
Build systems that interpret user intent, manage context across multi-turn interactions, and handle ambiguity reliably at scale
Develop and evolve agentic workflows including memory, context management, and multi-step tool orchestration
Create evaluation frameworks, including LLM-as-judge pipelines, to measure quality and guide iteration
Partner closely with product, engineering, and design to deliver seamless, user-facing experiences
Balance experimentation with production rigor, ensuring performance, latency, and reliability at Spotify scale
Continuously improve agent behavior through tight feedback loops between evaluation and real-world usage
Who You Are
5+ years of experience building and shipping machine learning systems in production environments
Experienced with large language models and have worked on real-world applications beyond experimentation; shipped and maintained large scale systems with LLMs
Deep understanding of challenges in conversational or agentic systems, such as context handling and multi-step reasoning
Know how to evaluate ML systems rigorously and have experience designing metrics or evaluation pipelines
Comfortable debugging complex interactions between models, tools, and system constraints like latency
Care about building reliable, scalable systems that deliver high-quality user experiences
Enjoy working cross-functionally and contributing to a collaborative, inclusive team environment
Skills
Machine LearningLLMsConversational AIAgentic AILlm-As-JudgeContext ManagementMulti-Step ReasoningEvaluation FrameworksProduction Ml Systems
Senior ML Engineer building observability, evaluation frameworks, and improvement loops for production agentic AI systems. Requires 5+ years production ML/LLM experience, strong grounding in agent design or evaluation, and hands-on work taking systems from prototype to scale.
216k – 270k/yr
On-site5+ YOEML Engineering
Senior Software Engineer, Agent Oversight
Scale AISan Francisco, CA +1
Build platform infrastructure for observing, evaluating, and improving production AI agents at scale. Requires 4+ years software engineering experience with ML/LLM systems, strong backend/distributed systems skills, and collaboration with ML engineers.
216k – 270k/yr
On-site5+ YOEML Engineering
Senior Software Engineer, AI Infrastructure
Ambient.aiRedwood City, CA
Build and optimize scalable AI infrastructure for real-time inference, evaluation, and continuous improvement of LLMs, LVMs, computer vision, and multimodal models on large-scale video data. Requires 4+ years production ML systems experience, strong Python skills, and expertise in inference optimization and model serving.
168k – 205k/yr
Hybrid4+ YOEML Engineering
Senior Software Engineer
SpotOnChicago, IL +3
Senior AI Engineer building agentic workflows and orchestration layers to automate manual business processes for operations, sales, and customer support teams. Requires 7+ years software engineering experience, production LLM/agent expertise, and strong Python/TypeScript skills.
160k – 190k/yr
Hybrid7+ YOEML Engineering
Senior AI Engineer, Enterprise
SnowflakeMenlo Park, CA
Senior AI Engineer building scalable backend services, distributed systems, and reusable AI platform components (agent frameworks, evaluation pipelines, SDKs) that power Snowflake's internal agentic applications for GTM teams. Requires 5+ years software engineering experience, strong Python/FastAPI skills, and production LLM experience.