# Member of Technical Staff, Inference

**Company:** [xAI](https://hotfix.jobs/companies/xai)
**Location:** Palo Alto, CA
**Role:** ML Engineering
**Salary:** $180k – $440k/yr
**Experience:** 5+ years
**Skills:** C++, Rust, Gpu Kernels, vLLM, Sglang, Triton, Tensorrt-Llm, Quantization, Speculative Decoding, CI/CD, Load Balancing, Distributed Systems
**Posted:** 2026-07-06

> Build and optimize the high-performance inference platform serving Grok at massive scale. Design distributed serving infrastructure, low-level GPU optimizations, quantization, speculative decoding, and CI/CD for production reliability and low latency.

## Job Description

## Responsibilities
- Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).
- Optimize latency and throughput of model inference under real production workloads.
- Build reliable, high-concurrency serving systems that serve billions of users with 100% uptime, 0% error rate, and excellent tail latency.
- Benchmark, fine-tune, and accelerate inference engines (including low-level GPU kernel work and code generation).
- Develop custom tools to trace, replay, and fix issues across the full stack — from orchestration down to GPU kernels.
- Create robust CI/CD infrastructure for seamless endpoint deployment, image publishing, and inference engine updates.
- Accelerate research on scaling test-time compute, RL rollout, and model-hardware co-design for next-generation systems.

## Requirements
- Deep low-level systems programming (C/C++ or Rust).
- Experience with large-scale, high-concurrent production serving.
- Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.).
- Strong background in system optimizations: batching, caching, load balancing, parallelism.
- Low-level inference optimizations: GPU kernels, code generation.
- Algorithmic inference optimizations: quantization, speculative decoding, distillation, low-precision numerics.
- Experience with testing, benchmarking, and reliability of inference services.
- Experience designing and implementing CI/CD infrastructure for inference.

## Similar roles

- [Member of Technical Staff](https://hotfix.jobs/jobs/dd7c8bc4-741a-45d5-91a8-e6b1c4ad67e0) - xAI - Palo Alto, CA - $180k – $440k/yr
- [Member of Technical Staff, Applied Research](https://hotfix.jobs/jobs/d19460df-bf56-4f87-aa95-5e7bdf40798f) - LlamaIndex - San Francisco, CA - $180k – $250k/yr
- [Member of Technical Staff](https://hotfix.jobs/jobs/544a5b36-3a56-45fa-89fe-0873120ece22) - Black Forest Labs - San Francisco, CA - $180k – $290k/yr
- [Member of Technical Staff - X Search](https://hotfix.jobs/jobs/2480c779-d772-4053-b432-64b3e48d4cd3) - xAI - Palo Alto, CA - $180k – $440k/yr
- [Member of Technical Staff - Post-Training and RL](https://hotfix.jobs/jobs/ed7e3533-4b73-4672-b792-579d9520ae65) - xAI - Palo Alto, CA - $180k – $600k/yr

**Apply:** https://hotfix.jobs/jobs/70280e8d-9e99-4be1-a488-b13edc2d4080
**Canonical:** https://hotfix.jobs/jobs/70280e8d-9e99-4be1-a488-b13edc2d4080