# Staff Machine Learning Engineer, Voice AI

**Company:** [Together AI](https://hotfix.jobs/companies/together-ai)
**Location:** San Francisco, CA
**Role:** ML Engineering
**Salary:** $220k – $280k/yr
**Experience:** 8+ years
**Skills:** Python, PyTorch, Tensorrt-Llm, Sglang, vLLM, CUDA, Gpu Optimization, Asr, Tts, Speech-To-Text, Text-To-Speech, Audio Signal Processing, Snac, Encodec, Model Serving
**Posted:** 2026-05-19

> Staff ML Engineer to own the model serving stack for real-time voice inference (STT, TTS, speech-to-speech) on H100/H200 GPUs. Drive latency/throughput optimization using TRT-LLM and SGLang for models like Whisper and Parakeet.

## Job Description

## Responsibilities
- Own the voice inference roadmap end-to-end — define and execute the technical strategy for optimizing STT, TTS, and speech-to-speech models across Together's infrastructure
- Drive best-in-class inference performance — architect and implement systems targeting leading TTFB, throughput, and GPU utilization for voice workloads
- Lead productionization of voice models at scale — design the serving architecture for serverless and dedicated endpoints, including batching strategies, streaming inference pipelines, and memory management tailored to real-time audio
- Build the voice evaluation platform — design a rigorous evaluation framework covering WER across accents, languages, and noise conditions for STT; naturalness, latency, and pronunciation fidelity for TTS
- Shape the architecture for next-generation model support — anticipate and enable emerging model paradigms (audio-native LLMs, codec-based architectures, end-to-end speech-to-speech systems)
- Serve as the technical DRI for model partner integrations — lead collaboration with partners such as Cartesia, Deepgram, and Rime
- Diagnose and resolve performance problems — conduct systematic profiling and root-cause analysis from GPU kernel behavior to framework-level bottlenecks
- Influence platform architecture — partner with platform engineering leadership to ensure the serving layer meets latency and reliability demands of real-time voice APIs
- Define and scale voice fine-tuning capabilities — lead technical direction for enabling customers to fine-tune STT and TTS models on Together's infrastructure

## Requirements
- 8+ years of ML engineering experience with focus on model serving, inference optimization, or ML infrastructure at production scale
- Deep expertise in LLM serving engines (vLLM, SGLang, TensorRT-LLM)
- Expert-level Python and PyTorch proficiency with strong command of GPU optimization (CUDA kernels, memory hierarchies, profiling toolchains)
- Proven system design judgment and architectural decisions that held up at scale
- Strong technical leadership with high autonomy
- Sharp product intuition for developer tooling
- Strong foundation in speech and audio ML (ASR/TTS architectures, audio signal processing) preferred
- Familiarity with audio codec and tokenization schemes (SNAC, Encodec, DAC) is a plus
- Experience training or fine-tuning speech models at scale is an advantage
- Bachelor's or Master's in Computer Science, Electrical Engineering, or related field

## Nice-to-Haves
- Experience modifying engine internals and contributing improvements back to serving frameworks
- Proven ability to move fast in ambiguous, early-stage environments

## Similar roles

- [Member of Technical Staff — Audio and Voice AI](https://hotfix.jobs/jobs/be841ac6-7856-43df-b21f-0540dd7e0f37) - Stuut - San Francisco, CA - $220k – $320k/yr
- [Member of Technical Staff](https://hotfix.jobs/jobs/344874a8-2e4c-4d32-996a-d7003be82d05) - Perplexity - San Francisco, CA - $220k – $405k/yr
- [Member of Technical Staff](https://hotfix.jobs/jobs/a463067e-4ce4-4632-ac39-cbc9264db356) - Perplexity - San Francisco, CA - $220k – $405k/yr
- [Member of Technical Staff](https://hotfix.jobs/jobs/184e2951-ec23-4f0f-ae88-0596e3b8325e) - Perplexity - San Francisco, CA - $220k – $405k/yr
- [Senior Staff ML Engineer](https://hotfix.jobs/jobs/34877dab-3f17-4d89-bbca-0cc524a3836a) - Typeface - Palo Alto, CA - $220k – $247k/yr

**Apply:** https://hotfix.jobs/jobs/541b5ffa-0699-403d-aef8-abe340656188
**Canonical:** https://hotfix.jobs/jobs/541b5ffa-0699-403d-aef8-abe340656188