# Audio Inference Engineer, Model Efficiency

**Company:** [Cohere](https://hotfix.jobs/companies/cohere)
**Location:** Remote
**Role:** ML Engineering
**Skills:** C++, Python, PyTorch, TensorFlow, Gpu Programming, vLLM, Sglang, Tensorrt-Llm, Deep Learning, Transformers
**Posted:** 2025-11-07

> Develops high-performance audio inference systems, optimizing latency, throughput, and quality for real-time streaming workloads. Requires expertise in C++, Python, and deep learning models for audio/speech, with collaboration across training and serving teams.

## Job Description

## Responsibilities
- Advance core audio model serving metrics, including latency, throughput, and quality.
- Dive deep into systems, identify bottlenecks, and deliver creative solutions for audio processing and streaming workloads.
- Collaborate closely with training and serving infrastructure teams for seamless integration between model development and deployment.
- Special focus on real-time and streaming audio inference.

## Requirements
- Significant experience developing high-performance audio or machine learning inference systems.
- Proficiency with programming languages such as **C++** and **Python**.
- Hands-on experience with deep learning models for audio, speech, or language applications.
- Bias for action and strong results-oriented mindset.

## Nice-to-Haves
- GPU programming, low-level system optimization, model parallelization techniques over multiple GPUs.
- Experience with duplex real-time streaming architectures.
- Internals of machine learning frameworks for audio (**PyTorch**, **TensorFlow**, or specialized audio libraries).
- Experience with inference frameworks like **vLLM**, **SGLang**, **TensorRT-LLM**, or custom distributed inference systems.
- Sequence modeling (e.g., transformers for audio/speech) and end-to-end audio pipeline optimization.

## Similar roles

- [CoDesign & NextGen Performance Engineer](https://hotfix.jobs/jobs/2cb728b7-a9ea-44b0-b16a-47364021f7be) - Cerebras Systems - Sunnyvale, CA
- [Research Engineer, Privacy](https://hotfix.jobs/jobs/3220c5d0-0e5f-4869-b133-1ed126b4208a) - OpenAI - San Francisco, CA - $380k – $445k/yr
- [Research Engineer](https://hotfix.jobs/jobs/b4d18018-e64d-4828-a8b4-303225328e17) - Console - San Francisco, CA - $200k – $350k/yr
- [Software Engineer, AI Platform](https://hotfix.jobs/jobs/7ddad000-3e1c-47cb-a45b-31d8e72abc4f) - Notion - San Francisco, CA - $180k – $201k/yr
- [Machine Learning Engineer](https://hotfix.jobs/jobs/97713038-58cb-46c5-a9c7-0e1554e1ba4f) - Liftoff - Remote - $215k – $275k/yr

**Apply:** https://hotfix.jobs/jobs/c5f5263a-25d6-4843-b70d-d6d426826786
**Canonical:** https://hotfix.jobs/jobs/c5f5263a-25d6-4843-b70d-d6d426826786