Member of Technical Staff — Model Optimization and Inference

What You'll Do

• Own end-to-end inference optimization across our model stack — LLMs, audio models, and diffusion-based components
• Implement and tune KV cache strategies for long-context conversations, including eviction policies, compression, and memory-efficient attention
• Evaluate, deploy, and extend inference serving frameworks (vLLM, SGLang, TensorRT-LLM, etc.) for our specific workloads
• Profile and benchmark end-to-end latency and throughput; identify and systematically eliminate bottlenecks
• Build internal tooling that makes optimization work faster and more rigorous — profiling viewers, end-to-end inference test harnesses, and other infrastructure that helps the team move quickly
• Accelerate diffusion model inference — consistency models, step distillation, caching strategies, and custom kernel optimizations
• Apply and develop quantization techniques (INT8, INT4, GPTQ, AWQ, and beyond) to reduce memory footprint and increase throughput without meaningfully degrading quality
• Work closely with research and infrastructure to ensure new models ship with optimized serving from day one

What We're Looking For

• Deep expertise in LLM inference optimization — you've worked on KV caching, memory layout, attention kernels, or batching strategies in a production or research context
• Proficiency with inference serving frameworks — vLLM, SGLang, TensorRT-LLM, or similar — including the ability to go beyond default configurations and adapt them to non-standard use cases
• Experience optimizing diffusion model inference (latency reduction, step distillation, caching, or kernel-level work)
• Strong Python and PyTorch skills; comfort reading and writing CUDA or Triton kernels is a significant plus
• A systematic approach to profiling and optimization — you measure first, then optimize
• Familiarity with speculative decoding or other inference-time acceleration techniques

Bonus Points

• Hands-on experience with post-training quantization (GPTQ, AWQ, or similar) and understanding of quality/performance tradeoffs
• Familiarity with multimodal or streaming inference architectures
• Experience deploying real-time AI systems with hard latency SLAs
• Prior work at an AI lab, inference startup, or on a high-traffic model serving platform
• Contributions to open-source inference frameworks

Compensation

• $250,000 – $350,000 base salary, plus meaningful equity
• Health: HSA plan with ~$2,000 in company contributions
• PTO: 15 days + public holidays, and we close for a full week over the holidays
• Lunch, beverages, and snacks provided every workday
• Commuter benefits
• 401K: In the works