# Member of Technical Staff

**Company:** [Modal](https://hotfix.jobs/companies/modal)
**Location:** New York, NY, San Francisco, CA
**Role:** ML Engineering
**Salary:** $150k – $350k/yr
**Skills:** Llm Inference, Speculative Decoding, Disaggregated Prefill/Decode, Quantization, Fp8, Int4, Kv-Cache, Memory Management, Autoscaling, Flash Attention, Sglang, Kernels, Schedulers
**Posted:** 2026-07-05

> Member of Technical Staff conducting hands-on LLM inference research at Modal. Own end-to-end bets on techniques like speculative decoding, quantization, KV-cache management, and disaggregation to improve cost per token and tail latency on production workloads. Requires strong LLM serving stack expertise and a track record shipping research or systems.

## Job Description

## What you'll do
- Own end-to-end inference research bets: speculative decoding, disaggregated prefill/decode, quantization (FP8, INT4), KV-cache and memory management, autoscaling for spiky serverless traffic, and whatever else the research agenda calls for.
- Train custom speculators against real production traffic and feed what you learn back into target models -- acceptance length is the metric that decides the win.
- Work directly with customers alongside our Forward Deployed Engineers to deploy and tune models, and bring what you learn back into the research.
- Carry and expand collaborations with outside research labs, for example: our work with ZLab on DFlash, a speculator design built on KV injection and blockwise parallel drafting; our work with SGLang on specdec and multimodal inference performance; our work on Flash Attention 4 kernels.
- Work with engineering to turn frontier serving techniques into products: primitives for disaggregation, fast weight refresh for models that keep training after deployment, observability for quality and latency in production, or even a next-generation inference engine.
- Help shape the research agenda.

## Requirements
- A research-leaning or systems background in LLM inference, with work you can point to.
- Fluency in the LLM serving stack, from kernels and quantization up to schedulers and autoscaling.
- A record of shipping research or systems that other people build on, whether in a lab or in industry.
- The drive to independently take a research bet from idea to result, working in the open with the rest of the team.
- Ability to work in-person, in our NYC or San Francisco office.

## Similar roles

- [Member of Technical Staff, Machine Learning](https://hotfix.jobs/jobs/0690a304-2c3d-4cff-aa6b-d101ed35ca25) - Sieve - San Francisco, CA - $150k – $350k/yr
- [Member of Technical Staff, Model Training](https://hotfix.jobs/jobs/2970f6fe-48c0-4ad2-bb34-51d7b2b4b4f4) - Parallel - California - $150k – $300k/yr
- [Member of Technical Staff, Search Ranking](https://hotfix.jobs/jobs/42567fd9-0b81-4e32-978b-09f4bf114749) - Parallel - $150k – $300k/yr
- [Staff Software Engineer, Engineering AI Team](https://hotfix.jobs/jobs/518f3095-1264-413d-8cc7-7dca7b4bac0c) - Second Nature - Remote - $150k – $180k/yr
- [Member of Technical Staff - Voice Model](https://hotfix.jobs/jobs/8c4661b6-322b-46b1-b2a0-05c09a885bd3) - xAI - Palo Alto, CA - $150k – $450k/yr

**Apply:** https://hotfix.jobs/jobs/0be719d4-4a8a-484f-ad6e-6b8b60347618
**Canonical:** https://hotfix.jobs/jobs/0be719d4-4a8a-484f-ad6e-6b8b60347618