# Member of Technical Staff — Model Optimization and Inference

**Company:** [Nuance Labs](https://hotfix.jobs/companies/nuance-labs)
**Location:** Seattle, WA
**Role:** ML Engineering
**Salary:** $200k – $300k/yr
**Experience:** 0+ years
**Skills:** Python, PyTorch, vLLM, Sglang, Tensorrt-Llm, CUDA, Triton, Kv Caching, Quantization, Model Optimization
**Posted:** 2026-06-11

> Early-career engineer optimizing inference for real-time multimodal AI avatars. Focus on KV cache strategies, serving frameworks, quantization, and latency reduction for LLMs and diffusion models.

## Job Description

## What You’ll Do
- Contribute to end-to-end inference optimization across our model stack — LLMs, audio models, and diffusion-based components
- Implement and tune KV cache strategies for long-context conversations, including eviction policies, compression, and memory-efficient attention
- Work with inference serving frameworks (vLLM, SGLang, TensorRT-LLM, etc.) and extend them for our specific workloads
- Profile and benchmark end-to-end latency and throughput; identify and systematically eliminate bottlenecks
- Build internal tooling that makes optimization work faster and more rigorous — profiling viewers, end-to-end inference test harnesses, and other infrastructure that helps the team move quickly
- Accelerate diffusion model inference — consistency models, step distillation, caching strategies, and custom kernel optimizations
- Apply quantization techniques (INT8, INT4, GPTQ, AWQ, and beyond) to reduce memory footprint and increase throughput without meaningfully degrading quality
- Work closely with research and infrastructure to ensure new models ship with optimized serving from day one

## What We’re Looking For
- BS, MS, or PhD in CS, ML, or a related field — completed or in the final stretch
- Strong fundamentals in LLM inference or ML systems — KV caching, memory layout, attention kernels, batching, or serving — picked up through coursework, research, internships, or open-source
- Exposure to inference serving frameworks (vLLM, SGLang, TensorRT-LLM, or similar) — even at a research or hobby level
- Strong Python and PyTorch skills; familiarity with CUDA or Triton is a significant plus
- A systematic approach to profiling and optimization — you measure first, then optimize
- Curiosity about diffusion inference, speculative decoding, quantization, or other inference-time acceleration techniques

## Bonus Points
- Internship or research experience with LLM inference, ML systems, or model serving
- Contributions to open-source inference frameworks (vLLM, SGLang, TensorRT-LLM, etc.)
- CUDA / Triton kernel work, even at a research or hobby scale
- Publications or research projects in MLSys, model compression, or inference optimization
- Familiarity with multimodal or streaming inference architectures
- Experience with hard latency SLAs in any real-time system

## Compensation
$200,000 – $300,000 base salary, plus meaningful equity.

## Benefits
- Health: HSA plan with ~$2,000 in annual company contributions
- Time off: 15 days of PTO plus public holidays, and we close the office for a full week at year-end
- Food: Lunch, drinks, and snacks on us every workday
- Commuter benefits
- 401(k): In the works

## Similar roles

- [Senior / Staff AI Platform Engineer](https://hotfix.jobs/jobs/65f2c893-cd67-482a-9cfb-623b499605ee) - Clear Street - Remote - $200k – $350k/yr
- [Senior / Staff AI Model Engineer](https://hotfix.jobs/jobs/efc2964d-f187-4b13-b8a3-2e78080490a2) - Clear Street - Remote - $200k – $350k/yr
- [Staff Software Engineer, Agents](https://hotfix.jobs/jobs/353f3eaf-bf6e-4405-92aa-c39af858b5ac) - Decagon - San Francisco, CA - $200k – $400k/yr
- [Member of Technical Staff - Research Fellow](https://hotfix.jobs/jobs/319ee852-b158-4ee4-b29a-a680b8bfcfba) - Nuance Labs - Seattle, WA - $200k – $250k/yr
- [Staff Applied ML Scientist](https://hotfix.jobs/jobs/3a6196f5-a16a-432c-a4b9-3e5de534e15c) - SentiLink - Remote - $200k – $240k/yr

**Apply:** https://hotfix.jobs/jobs/d8ff8ff7-611a-4edc-94d4-c249e9aed787
**Canonical:** https://hotfix.jobs/jobs/d8ff8ff7-611a-4edc-94d4-c249e9aed787