# Senior Machine Learning Engineer

**Company:** [Ambience Healthcare](https://hotfix.jobs/companies/ambience-healthcare)
**Location:** San Francisco, CA
**Role:** ML Engineering
**Salary:** $225k – $300k/yr
**Experience:** 5+ years
**Skills:** Python, PyTorch, LLMs, Transformers, Machine Learning, NLP, Retrieval, Fine-Tuning, Active Learning, CI/CD, Docker, Observability, Agentic AI, Multimodal Ai, Conversational AI
**Posted:** 2026-08-11

> Build and improve production AI systems for clinical products, owning evaluations, model behavior, agentic workflows, data flywheels, deployment, and observability. The role requires 5+ years of production ML or applied AI experience, strong Python and modern ML framework skills, and hands-on debugging expertise.

## Job Description

## Responsibilities
- Design and own evaluation pipelines for LLM and agentic systems using automated graders, regression testing, production feedback, and human evaluation.
- Diagnose production failure modes and improve model behavior through prompting, retrieval, context, routing, data, fine-tuning, and other system interventions.
- Build production agentic AI systems with tool use, retrieval, context and state management, routing, orchestration, tracing, and failure recovery.
- Create datasets, evaluations, and feedback loops using production failures, user feedback, and active learning.
- Translate research in LLMs, agents, NLP, speech, and multimodal AI into practical experiments.
- Own AI systems end-to-end across models, data, evaluation, orchestration, serving, and observability.
- Collaborate with clinicians, product managers, and engineers; communicate complex AI concepts clearly.

## Requirements
- 5+ years of experience in production machine learning, research engineering, or applied AI.
- Experience building a consequential production AI system or materially improving model behavior in production.
- Strong understanding of modern LLMs, transformers, and production AI systems.
- Experience designing evaluations for LLMs, agents, or other complex AI systems.
- Ability to turn ambiguous quality problems into measurable dimensions, datasets, and experiments.
- Familiarity with grader bias, leakage, misleading aggregate metrics, regression detection, and offline-online mismatch.
- Experience building production systems involving multiple models, tools, retrieval, context, state, routing, or orchestration.
- Proficiency in Python and modern machine learning frameworks; PyTorch preferred.
- Comfort with deployment, observability, CI/CD, and containerized systems.
- Hands-on experience writing code, inspecting traces, analyzing failures, and debugging production systems.
- Experience building high-quality datasets and feedback loops.
- Strong interdisciplinary collaboration, communication, ownership, and problem-solving skills.

## Nice-to-Haves
- Experience with real-time voice, conversational AI, or multimodal systems.
- Experience with fine-tuning, post-training, or model adaptation.
- Experience in healthcare, clinical AI, or other regulated, high-stakes industries.
- Experience interviewing or mentoring machine learning engineers.
- Open-source contributions to machine learning, agent, or evaluation tooling.

## Compensation and Benefits
- $225,000–$300,000, plus significant equity.
- Comprehensive medical, dental, and vision coverage for employees and dependents.
- 401(k) with company match of up to 3% of base salary.
- Remote-friendly culture with a San Francisco headquarters and equipment provisioning.
- Parental leave.
- Annual company-wide and team off-sites, team lunches, and all-hands gatherings with travel, lodging, and meals covered.
- Flexible time off with no annual cap, company holidays, and an annual holiday shutdown from December 24–January 1.

## Similar jobs

- [Senior AI Engineer](https://hotfix.jobs/jobs/159f025f-dfce-49a3-907d-f922fdd04400) - Dialpad - Remote - $225k – $256k/yr
- [Senior AI Engineer, Agentic Data Enrichment](https://hotfix.jobs/jobs/5246cc35-17f3-4041-a991-5cda61fcb0cf) - Baselayer - San Francisco, CA - $230k – $340k/yr
- [Senior Applied Scientist, Speech](https://hotfix.jobs/jobs/fc2161b1-228d-49f4-ad53-dc2f66db10a8) - Otter - Mountain View, CA - $230k – $265k/yr
- [Senior Machine Learning Engineer, Ads](https://hotfix.jobs/jobs/38a90e06-7152-474a-aaf0-bd106f198aee) - Reddit - Remote - $217k – $303k/yr
- [Senior Machine Learning Engineer II](https://hotfix.jobs/jobs/f137b594-66ba-4853-b8de-a4705ab39ba6) - Fetch - Remote - $211k – $249k/yr

**Apply:** https://hotfix.jobs/jobs/c2b04718-b102-4765-ab70-ecbc4e50104e
**Canonical:** https://hotfix.jobs/jobs/c2b04718-b102-4765-ab70-ecbc4e50104e