Member of Technical Staff conducting hands-on LLM inference research at Modal. Own end-to-end bets on techniques like speculative decoding, quantization, KV-cache management, and disaggregation to improve cost per token and tail latency on production workloads. Requires strong LLM serving stack expertise and a track record shipping research or systems.
150k – 350k/yr
On-siteML Engineering
About the role
What you'll do
Own end-to-end inference research bets: speculative decoding, disaggregated prefill/decode, quantization (FP8, INT4), KV-cache and memory management, autoscaling for spiky serverless traffic, and whatever else the research agenda calls for.
Train custom speculators against real production traffic and feed what you learn back into target models -- acceptance length is the metric that decides the win.
Work directly with customers alongside our Forward Deployed Engineers to deploy and tune models, and bring what you learn back into the research.
Carry and expand collaborations with outside research labs, for example: our work with ZLab on DFlash, a speculator design built on KV injection and blockwise parallel drafting; our work with SGLang on specdec and multimodal inference performance; our work on Flash Attention 4 kernels.
Work with engineering to turn frontier serving techniques into products: primitives for disaggregation, fast weight refresh for models that keep training after deployment, observability for quality and latency in production, or even a next-generation inference engine.
Help shape the research agenda.
Requirements
A research-leaning or systems background in LLM inference, with work you can point to.
Fluency in the LLM serving stack, from kernels and quantization up to schedulers and autoscaling.
A record of shipping research or systems that other people build on, whether in a lab or in industry.
The drive to independently take a research bet from idea to result, working in the open with the rest of the team.
Ability to work in-person, in our NYC or San Francisco office.
Machine Learning Engineer owning the full ML lifecycle for multimodal video datasets at Sieve. Fine-tune VLMs, build evaluation/QA pipelines with frontier models, design filtering systems over internet-scale data, and ship production improvements for top AI labs. Requires strong Python, PyTorch, and production ML experience.
150k – 350k/yr
On-site5+ YOEML Engineering
Member of Technical Staff, Model Training
ParallelCalifornia
Own the training pipeline for search and agent models, building from product usage data through fine-tuning and evaluation to production deployment. Requires deep expertise in transformer fine-tuning, data curation, and training models for ranking, retrieval, and agent behavior.
150k – 300k/yr
On-siteML Engineering
Member of Technical Staff, Search Ranking
ParallelUnited States
Own the multi-stage ranking pipeline for web-scale search, balancing precision, recall, latency, and compute cost across retrieval, first-pass ranking, and neural reranking.
150k – 300k/yr
On-site7+ YOEML Engineering
Staff Software Engineer, Engineering AI Team
Second NatureUnited States
Staff engineer builds AI-driven platform infrastructure for SDLC transformation, owns end-to-end experiments using AI agents like Claude, and ensures high-velocity code delivery with strong abstractions and real-world grounding. Requires staff-level architecture experience and AI-native workflows.
150k – 180k/yr
RemoteML Engineering
Member of Technical Staff - Voice Model
xAIPalo Alto, CA
Develop voice AI models for natural, low-latency spoken interactions on the Grok team. Handle data pipelines, model training with JAX/PyTorch, evaluations, and product integrations. Requires Python expertise, large-scale data processing, and distributed systems experience.