Senior Machine Learning Engineer, Voice AI
Senior ML Engineer optimizes inference for voice AI models (STT, TTS, speech-to-speech) using engines like TensorRT-LLM and SGLang on GPUs. Requires 5+ years ML engineering with serving/inference expertise, Python/PyTorch proficiency, and production ML experience.
About the job
Responsibilities
- Optimize inference performance for voice models (STT, TTS, speech-to-speech) — targeting best-in-class TTFB, throughput, and GPU utilization across curated model set.
- Productionize voice models on serverless and dedicated endpoints, including batching strategies, streaming inference, and memory management tailored to audio workloads.
- Build and maintain a voice model evaluation framework — measuring WER across accents, languages, and noise conditions for STT; naturalness, latency, and pronunciation accuracy for TTS.
- Enable new model architectures in serving stack as field evolves, including audio-native LLMs, codec-based models (SNAC), and speech-to-speech systems.
- Collaborate with model partners to integrate and optimize their models (Cartesia, Deepgram, Rime, and others) running on Together's infrastructure.
- Profile and debug performance across full inference stack — from GPU kernels to framework-level bottlenecks — and ship measurable improvements.
- Work with platform engineering to ensure serving layer meets latency and reliability requirements of real-time voice APIs.
- Contribute to voice model fine-tuning capabilities (STT and TTS) as customers build differentiated voice experiences.
- Lay groundwork for multiple new products.
Requirements
- 5+ years of experience in ML engineering, with focus on model serving, inference optimization, or ML infrastructure.
- Hands-on experience with LLM serving engines (vLLM, SGLang, TensorRT-LLM, or similar) — comfortable reading and modifying engine internals.
- Strong proficiency in Python and PyTorch; experience with GPU profiling and optimization (CUDA, memory management, kernel-level debugging).
- Track record of shipping ML systems to production with measurable performance improvements.
- Strong product sense — understanding developer needs for voice apps.
- Comfort working on small, early-stage team, wearing multiple hats and moving fast.
Nice-to-Haves
- Experience with speech and audio ML (ASR, TTS architectures, audio signal processing).
- Familiarity with audio codecs and tokenization schemes (SNAC, Encodec, DAC).
- Experience training or fine-tuning speech models.
- Bachelor's or Master's degree in Computer Science, Electrical Engineering, or related field, or equivalent practical experience.
Compensation
US base salary range: $200,000 - $260,000 + equity + benefits.
Skills
PyTorch, Python, Tensorrt-Llm, vLLM, Sglang, CUDA, Whisper, Trt-Llm, Snac, Gpu Optimization
Similar jobs
ML Engineering jobs
Traba
New York, NYBuild and deploy production AI-agent systems, including their harnesses, evaluations, orchestration, and supporting services. The role requires 5+ years of software engineering experience, production LLM or agent experience, and strong Python or TypeScript/Node.js skills.
Metriport
San Francisco, CA
Own machine learning end to end, from modeling messy clinical data through production deployment, monitoring, and infrastructure. The role requires 7+ years of experience building scalable ML systems, strong software and data engineering skills, and proficiency with Python, SQL, and cloud platforms.
SentiLink
United States
Leads and manages an applied machine learning team developing production fraud detection and identity verification models. The role combines people leadership with hands-on technical work and requires substantial ML experience, production deployment expertise, and experience in risk-focused domains.
Airbnb
United States
Builds and productionizes machine learning systems for trust and safety, including abuse detection, autonomous AI agents, and evaluation frameworks. The role requires 5+ years of applied ML experience, strong Python skills, experience with LLMs and scalable pipelines, and a relevant advanced degree or equivalent background.
6sense
San Francisco, CA
Owns end-to-end production machine learning systems, including NLP, LLM, agentic, ranking, and recommendation capabilities. Requires 8+ years of industry experience, strong Python and cloud ML expertise, and the ability to deliver explainable AI products with cross-functional and customer impact.