Member of Technical Staff — Model Optimization and Inference

What You’ll Do

• Contribute to end-to-end inference optimization across our model stack — LLMs, audio models, and diffusion-based components
• Implement and tune KV cache strategies for long-context conversations, including eviction policies, compression, and memory-efficient attention
• Work with inference serving frameworks (vLLM, SGLang, TensorRT-LLM, etc.) and extend them for our specific workloads
• Profile and benchmark end-to-end latency and throughput; identify and systematically eliminate bottlenecks
• Build internal tooling that makes optimization work faster and more rigorous — profiling viewers, end-to-end inference test harnesses, and other infrastructure that helps the team move quickly
• Accelerate diffusion model inference — consistency models, step distillation, caching strategies, and custom kernel optimizations
• Apply quantization techniques (INT8, INT4, GPTQ, AWQ, and beyond) to reduce memory footprint and increase throughput without meaningfully degrading quality
• Work closely with research and infrastructure to ensure new models ship with optimized serving from day one

What We’re Looking For

• BS, MS, or PhD in CS, ML, or a related field — completed or in the final stretch
• Strong fundamentals in LLM inference or ML systems — KV caching, memory layout, attention kernels, batching, or serving — picked up through coursework, research, internships, or open-source
• Exposure to inference serving frameworks (vLLM, SGLang, TensorRT-LLM, or similar) — even at a research or hobby level
• Strong Python and PyTorch skills; familiarity with CUDA or Triton is a significant plus
• A systematic approach to profiling and optimization — you measure first, then optimize
• Curiosity about diffusion inference, speculative decoding, quantization, or other inference-time acceleration techniques

Bonus Points

• Internship or research experience with LLM inference, ML systems, or model serving
• Contributions to open-source inference frameworks (vLLM, SGLang, TensorRT-LLM, etc.)
• CUDA / Triton kernel work, even at a research or hobby scale
• Publications or research projects in MLSys, model compression, or inference optimization
• Familiarity with multimodal or streaming inference architectures
• Experience with hard latency SLAs in any real-time system

Compensation

$200,000 – $300,000 base salary, plus meaningful equity.

Benefits

• Health: HSA plan with ~$2,000 in annual company contributions
• Time off: 15 days of PTO plus public holidays, and we close the office for a full week at year-end
• Food: Lunch, drinks, and snacks on us every workday
• Commuter benefits
• 401(k): In the works