# Member of Technical Staff — Model Optimization and Inference

**Company:** [Nuance Labs](https://hotfix.jobs/companies/nuance-labs)
**Location:** Seattle, WA
**Role:** ML Engineering
**Salary:** $250k – $350k/yr
**Experience:** 7+ years
**Skills:** Python, PyTorch, vLLM, Sglang, Tensorrt-Llm, CUDA, Triton, Kv Cache Optimization, Quantization, Gptq, Awq, Diffusion Models, Speculative Decoding
**Posted:** 2026-06-05

> Optimize inference for real-time multimodal AI avatars. Specialize in LLM and diffusion model serving, KV cache strategies, quantization, and low-latency frameworks like vLLM and TensorRT-LLM.

## Job Description

## What You'll Do
- Own end-to-end inference optimization across our model stack — LLMs, audio models, and diffusion-based components
- Implement and tune KV cache strategies for long-context conversations, including eviction policies, compression, and memory-efficient attention
- Evaluate, deploy, and extend inference serving frameworks (vLLM, SGLang, TensorRT-LLM, etc.) for our specific workloads
- Profile and benchmark end-to-end latency and throughput; identify and systematically eliminate bottlenecks
- Build internal tooling that makes optimization work faster and more rigorous — profiling viewers, end-to-end inference test harnesses, and other infrastructure that helps the team move quickly
- Accelerate diffusion model inference — consistency models, step distillation, caching strategies, and custom kernel optimizations
- Apply and develop quantization techniques (INT8, INT4, GPTQ, AWQ, and beyond) to reduce memory footprint and increase throughput without meaningfully degrading quality
- Work closely with research and infrastructure to ensure new models ship with optimized serving from day one

## What We're Looking For
- Deep expertise in LLM inference optimization — you've worked on KV caching, memory layout, attention kernels, or batching strategies in a production or research context
- Proficiency with inference serving frameworks — vLLM, SGLang, TensorRT-LLM, or similar — including the ability to go beyond default configurations and adapt them to non-standard use cases
- Experience optimizing diffusion model inference (latency reduction, step distillation, caching, or kernel-level work)
- Strong Python and PyTorch skills; comfort reading and writing CUDA or Triton kernels is a significant plus
- A systematic approach to profiling and optimization — you measure first, then optimize
- Familiarity with speculative decoding or other inference-time acceleration techniques

## Bonus Points
- Hands-on experience with post-training quantization (GPTQ, AWQ, or similar) and understanding of quality/performance tradeoffs
- Familiarity with multimodal or streaming inference architectures
- Experience deploying real-time AI systems with hard latency SLAs
- Prior work at an AI lab, inference startup, or on a high-traffic model serving platform
- Contributions to open-source inference frameworks

## Compensation
- $250,000 – $350,000 base salary, plus meaningful equity
- Health: HSA plan with ~$2,000 in company contributions
- PTO: 15 days + public holidays, and we close for a full week over the holidays
- Lunch, beverages, and snacks provided every workday
- Commuter benefits
- 401K: In the works

## Similar roles

- [Staff Software Engineer, Agentic Platform](https://hotfix.jobs/jobs/63ce2244-1c65-4c7d-820a-ca8afb78bb60) - Axion - San Francisco, CA - $250k – $270k/yr
- [Member of Research Staff, Optimization](https://hotfix.jobs/jobs/cd745934-2bf2-444e-b0b7-dc259cf569a4) - The Voleon Group - Berkeley, CA - $250k – $275k/yr
- [Member of Technical Staff — RL Research](https://hotfix.jobs/jobs/72e143f9-9ecc-422b-b67d-d6c97094fbaa) - Nuance Labs - Seattle, WA - $250k – $350k/yr
- [Staff Software Engineer, AI Data Platform](https://hotfix.jobs/jobs/1914a72f-f09f-498a-8edf-51b951be967a) - Labelbox - San Francisco, CA - $250k – $280k/yr
- [Staff AI Engineer - AI Product](https://hotfix.jobs/jobs/c1f2fb81-d918-4abd-873b-9b108332280f) - ClickUp - Remote - $250k – $300k/yr

**Apply:** https://hotfix.jobs/jobs/c81df2dc-c760-4cc8-9c23-2aa36efed90e
**Canonical:** https://hotfix.jobs/jobs/c81df2dc-c760-4cc8-9c23-2aa36efed90e