AI Engineer, Agentic Voice
Build and optimize the backend TTS layer for next-generation AI agents, integrating speech vendors, improving latency and naturalness, and implementing persona systems. The role requires production Python, speech ML, linguistic optimization, cloud API, evaluation, and prompt-engineering experience.
About the job
Responsibilities
- Own the integration and optimization of multiple text-to-speech (TTS) vendor APIs behind a unified interface with failover.
- Lead research and prototyping of open-source and in-house TTS architectures.
- Minimize time-to-first-audio and end-to-end latency across the ASR → LLM → TTS pipeline while maintaining or improving voice quality.
- Apply phonetics and sociolinguistics to format TTS input for naturalness, including SSML tags, punctuation-driven prosody, and text normalization for names, numbers, dates, and currency.
- Build the persona parameterization system and expose voice attributes to the product UI.
- Manage structured LLM and TTS prompt templates with versioning and an evaluation harness.
- Engineer context- and state-aware thinking utterances that maintain caller trust during tool calls and model execution.
- Partner with ASR and Audio AI engineers and collaborate with Voice Experience Design.
Requirements
- Strong Python skills and hands-on experience with deep learning frameworks such as PyTorch.
- 3+ years of experience in speech synthesis (TTS) or applied speech machine learning.
- Hands-on experience with NVIDIA NeMo, ESPnet, or Coqui.
- Experience with TTS APIs such as ElevenLabs, Rime, or Cartesia.
- Understanding of phonetics, prosody, syntax, G2P, pronunciation dictionaries, and text normalization.
- Experience building production-grade APIs and integrating multi-vendor services in a cloud environment; GCP is preferred.
- Fluency with speech-quality metrics including MOS, intelligibility, and latency.
- Ability to design and execute rigorous A/B tests for voice personas.
- Proven experience crafting, versioning, and evaluating system, few-shot, and structured LLM prompts.
- Degree in Computational Linguistics, Computer Science, or AI/ML.
Compensation
- British Columbia, Canada base salary: $161,500–$191,500 CAD.
- Compensation details exclude bonus, equity, and benefits.
- Competitive benefits, training, and growth opportunities.
Skills
Python, PyTorch, Speech Synthesis, Nvidia Nemo, Espnet, Coqui, Elevenlabs, Rime, Cartesia, Phonetics, Prosody, G2P, GCP, Ssml, Prompt Engineering
Similar jobs
ML Engineering jobs
Instacart
Canada
Develop and deploy machine learning systems for Instacart’s advertising ecosystem, spanning data pipelines, model architectures, serving, experimentation, and optimization. The role requires a graduate degree and strong programming, analytical, and collaboration skills, with experience in large-scale ML systems preferred.
Coalition Security
United StatesBuild and operate LLM-powered agents that automate business workflows, integrating systems of record through MCP and validating behavior with structured evaluations. Requires 5+ years of software development experience, strong Python and SQL, and hands-on experience shipping agents to production.
Rollstack
United StatesBuild production AI capabilities for automated slide and document generation, working across LLM applications, data analysis, and content generation. The role requires 3+ years in machine learning and NLP, advanced Python, and experience with LLM frameworks and production systems.
Protege
Remote
Build the technical foundation for a new business vertical, creating reusable infrastructure and leading early customer engagements from scoping through delivery. The role requires 3+ years of engineering experience, strong Python and SQL skills, backend/data expertise, and comfort operating in ambiguity.
Adaption Labs
San Francisco, CA
Build production agent systems that plan, use tools, recover from failures, and improve over time. The role requires 5+ years of production ML or backend experience, LLM or agent deployment experience, and expertise in evaluation, tracing, observability, and agent architecture.