# Member of Technical Staff, Model Efficiency

**Company:** [Cohere](https://hotfix.jobs/companies/cohere)
**Location:** Remote
**Role:** ML Engineering
**Experience:** 5+ years
**Skills:** C++, Python, Rust, Go, LLMs, vLLM, Sglang, CUDA, GPU, Transformers, Moe, Kv-Cache
**Posted:** 2025-11-07

> Engineers on this team optimize LLM inference for lower latency and higher throughput by identifying bottlenecks, developing optimizations across the execution stack, and collaborating with modeling teams. Requires 5+ years high-performance coding in C++/Python and LLM inference experience.

## Job Description

## Responsibilities
- Work across the inference stack to improve core performance metrics by diving deep into model execution, identifying bottlenecks, and developing innovative optimizations.
- Collaborate closely with modeling and systems teams to experiment, measure, and ship improvements that meaningfully accelerate inference.
- Build expertise in advanced performance techniques, including GPU/CUDA optimizations, kernel-level improvements, and model execution strategies for MoE and large-scale architectures.

## Requirements
- 5+ years of experience writing high-performance, production-quality code.
- Strong programming skills in **C++** or **Python** (**Rust**/**Go** also welcome).
- Experience working with large language models and familiarity with the LLM inference ecosystem (e.g., **vLLM**, **SGLang**, etc.).
- Ability to diagnose and resolve performance bottlenecks across the model execution stack.
- A strong bias for action — ship fast, measure impact, and iterate.

## Nice-to-Haves
- Experience with **GPU** programming, **CUDA**, or low-level systems optimization.
- Language modeling with **transformers** (**MoE**, speculative decoding, **KV-cache** optimizations).
- Scaling performance-critical distributed systems (e.g., computation, search, storage).

## Similar roles

- [Member of Technical Staff](https://hotfix.jobs/jobs/dd7c8bc4-741a-45d5-91a8-e6b1c4ad67e0) - xAI - Palo Alto, CA - $180k – $440k/yr
- [Staff Software Engineer, AI & App Experience](https://hotfix.jobs/jobs/78128221-8199-41cd-9c63-502d3f61d603) - Chime - New York, NY - $223k – $308k/yr
- [Staff + Senior Software Engineer, Cloud Inference Launch Engineering](https://hotfix.jobs/jobs/18b08a6a-5359-4836-b72e-c641fa718b04) - Anthropic - San Francisco, CA - $320k – $485k/yr
- [Senior Staff Software Engineer, Legal Automation](https://hotfix.jobs/jobs/7870d3e1-ebec-4f7b-9ae5-c62926a06d6e) - Coinbase - Remote - $254k – $299k/yr
- [Staff Machine Learning Engineer, Computer Vision](https://hotfix.jobs/jobs/2969c5c0-8479-43bb-b9b6-f8df93b528d7) - Pinterest - San Francisco, CA - $189k – $390k/yr

**Apply:** https://hotfix.jobs/jobs/c60c8aee-7682-4764-b59d-a90f728b0954
**Canonical:** https://hotfix.jobs/jobs/c60c8aee-7682-4764-b59d-a90f728b0954