# Distributed Systems Engineer, Data & Inference Platform

**Company:** [Adaption Labs](https://hotfix.jobs/companies/adaption-labs)
**Location:** San Francisco, CA, Fremont, CA, Palo Alto, CA, Berkeley, CA, Sunnyvale, CA, Mountain View, CA, San Jose, CA, Oakland, CA, Redwood City, CA
**Role:** ML Engineering
**Experience:** 5+ years
**Skills:** Ray, Spark, Kubernetes, Python, Go, Rust, C++, CUDA, Nccl, vLLM, Tensorrt-Llm
**Posted:** 2026-05-07

> Builds and operates distributed inference systems for LLMs at scale and large-scale data pipelines for training/evaluation. Requires 5+ years in production distributed systems, GPU expertise, and frameworks like Ray/Spark.

## Job Description

## Responsibilities
- **Serve Models at Scale**: Design and operate distributed inference systems for LLMs, optimizing throughput, latency, and cost across heterogeneous GPU fleets. Batching, scheduling, KV cache management, autoscaling.
- **Move the Data**: Build large-scale data pipelines (Ray Data, Spark, or equivalents) that ingest, transform, and curate the datasets behind training and evaluation.
- **Debug the Undebuggable**: Chase down failure modes under production traffic — stragglers, head-of-line blocking, silent data corruption, GPU memory fragmentation — and write postmortems. Define SLOs, build observability, own on-call rotation.
- **Partner Across the Stack**: Work directly with researchers and ML engineers to productionize experimental workloads.

## Qualifications
- 5+ years building and operating distributed systems in production.
- Deep experience with at least one large-scale data or compute framework (Ray, Spark, Flink, Beam, Dask).
- Strong fluency in **Python** and at least one systems language (**Go**, **Rust**, **C++**).
- Working knowledge of the GPU/accelerator stack: **CUDA** fundamentals, **NCCL**, mixed precision, memory layout.
- Experience operating **Kubernetes**-based infrastructure, including custom operators or schedulers.
- Track record of owning hard production incidents end-to-end.

## Bonus
- Hands-on experience with LLM inference engines (**vLLM**, **SGLang**, **TensorRT-LLM**, **TGI**), modern lakehouse formats (**Iceberg**, **Delta**, **Hudi**), or open-source contributions to relevant projects.

## Benefits
- Flexible work: In-person collaboration in the Bay Area.
- Adaption Passport: Annual travel stipend.
- Lunch Stipend: Weekly meal allowance.
- Well-Being: Comprehensive medical benefits and generous paid time off.

## Similar roles

- [Software Engineer, AI Agents](https://hotfix.jobs/jobs/94686d85-8544-4cc4-a8cf-41545c3d9d95) - Cloudflare
- [CoDesign & NextGen Performance Engineer](https://hotfix.jobs/jobs/2cb728b7-a9ea-44b0-b16a-47364021f7be) - Cerebras Systems - Sunnyvale, CA
- [Research Engineer, Privacy](https://hotfix.jobs/jobs/3220c5d0-0e5f-4869-b133-1ed126b4208a) - OpenAI - San Francisco, CA - $380k – $445k/yr
- [Research Engineer](https://hotfix.jobs/jobs/b4d18018-e64d-4828-a8b4-303225328e17) - Console - San Francisco, CA - $200k – $350k/yr
- [Software Engineer, AI Platform](https://hotfix.jobs/jobs/7ddad000-3e1c-47cb-a45b-31d8e72abc4f) - Notion - San Francisco, CA - $180k – $201k/yr

**Apply:** https://hotfix.jobs/jobs/4df0a658-e99a-47d5-8f13-5c035913443a
**Canonical:** https://hotfix.jobs/jobs/4df0a658-e99a-47d5-8f13-5c035913443a