# Software Engineer - Voice AI (Inference Runtime)

**Company:** [Baseten](https://hotfix.jobs/companies/baseten)
**Location:** San Francisco, CA, New York, NY
**Role:** ML Engineering
**Salary:** $165k – $330k/yr
**Skills:** Python, Kubernetes, Docker, PyTorch, vLLM, TensorRT, Onnx, Whisper, Stt, Tts
**Posted:** 2026-04-23

> Build and own high-performance inference runtime for Voice AI models including STT, TTS, and voice agents. Design real-time systems with low tail latency, collaborate cross-team, and optimize model serving for production workloads. Requires CS degree and real-time systems experience.

## Job Description

## Responsibilities
- Own and lead Voice AI product areas end-to-end — from architecture and system design through implementation, rollout, and long-term production operations.
- Design, build, and operate real-time, large-scale, high-performance model serving systems for STT, TTS, and voice agent workloads with clear SLOs for mission-critical customer deployments.
- Drive cross-team collaboration with sister engineering teams to solve full-stack technical problems, aligning on priorities, and coordinating end-to-end delivery across the product surface area.
- Mentor teammates through code reviews, design docs, and technical leadership.

## Requirements
- Bachelor's degree or higher in Computer Science or related field
- Proven track record owning production-grade real-time, large-scale systems where tail latency (p99) matters.
- Proficient coding abilities in one or more popular programming or scripting languages; **Python** proficiency is a plus.
- Good taste in product, particularly developer-oriented tools
- Interest in ML/AI infrastructure and willingness to learn
- Strong collaboration and communication skills
- Comfortable using AI coding assistants (e.g., Claude Code, Codex, Cursor) as a daily productivity multiplier

## Nice to Have
- Experience implementing pipeline-level model runtime optimizations such as dynamic batching, async scheduling, or decode-side throughput improvements.
- Experience building developer platforms: SDKs, CLIs, APIs, and self-serve workflows for ML or infrastructure products.
- Experience with containerization and orchestration technologies (**Docker**, **Kubernetes**), service meshes, or distributed scheduling.
- Familiarity with speech/audio ML models (STT, TTS, speech-to-speech)
- Familiarity with model-serving runtimes (**vLLM**, **TensorRT**, **ONNX**).
- Familiarity with systems-level performance profiling across host-device boundaries (e.g. **PyTorch Profiler**), diagnosing GPU utilization issues
- Exposure to customer-facing engineering: pre-sales prototyping, technical discovery, or working directly with customers to ship solutions.

## Benefits
- Competitive compensation, including meaningful equity.
- 100% coverage of medical, dental, and vision insurance for employee and dependents
- Flexible PTO policy including company wide Winter Break
- Paid parental leave
- Fertility and family-building stipend through Carrot
- Company-facilitated 401(k)
- Exposure to a variety of ML startups

## Similar roles

- [Forward Deployed Engineer, Deepgram for Restaurants](https://hotfix.jobs/jobs/3e0f6a46-146c-4ecd-9d4a-f1a61427be5e) - Deepgram - Remote - $165k – $220k/yr
- [Software Engineer, AI/ML](https://hotfix.jobs/jobs/79829b86-da09-45a9-b27a-329f2888328a) - Tabs - New York, NY - $165k – $225k/yr
- [Fullstack Software Engineer, Applied AI](https://hotfix.jobs/jobs/557ac0c0-36f8-49b8-acb1-8973974334dc) - LangChain - New York, NY - $165k – $190k/yr
- [Software Engineer - Training Infrastructure](https://hotfix.jobs/jobs/8ab5aff7-f0c3-4cc7-8796-5c868b5098cd) - Baseten - San Francisco, CA - $165k – $330k/yr
- [Software Engineer, AI Growth](https://hotfix.jobs/jobs/2537186d-1094-4bd9-9078-4509772ccbba) - Benchling - San Francisco, CA - $165k – $223k/yr

**Apply:** https://hotfix.jobs/jobs/53a7ab9f-cf11-4bde-9646-c3bbdea3c0ad
**Canonical:** https://hotfix.jobs/jobs/53a7ab9f-cf11-4bde-9646-c3bbdea3c0ad