# Software Engineer, Inference

**Company:** [Luma AI](https://hotfix.jobs/companies/lumalabs-ai)
**Location:** Palo Alto, CA
**Role:** ML Engineering
**Salary:** $188k – $395k/yr
**Skills:** Python, PyTorch, Huggingface, vLLM, Sglang, Tensorrt-Llm, Kubernetes, Docker, Linux, Redis
**Posted:** 2026-01-21

> Develops and optimizes inference engines for multimodal AI models, integrating new architectures, building scheduling systems, and managing large-scale GPU deployments. Requires strong Python, model serving frameworks like PyTorch/vLLM, and Kubernetes expertise.

## Job Description

## Role & Responsibilities
- Ship new model architectures by integrating them into our inference engine
- Collaborate closely across research, engineering and infrastructure to streamline and optimize model efficiency and deployments
- Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows
- Automate, test and maintain our inference services to ensure maximum uptime and reliability
- Optimize deployment workflows to scale across thousands of machines
- Manage and optimize our inference workloads across different clusters & hardware providers
- Build sophisticated scheduling systems to optimally leverage our expensive GPU resources while meeting internal SLOs
- Build and maintain CI/CD pipelines for processing/optimizing model checkpoints, platform components, and SDKs for internal teams to integrate into our products/internal tooling

## Background
**Must have:**
- Strong Python and system architecture skills
- Experience with model deployment using PyTorch, Huggingface, vLLM, SGLang, tensorRT-LLM, or similar
- Experience with queues, scheduling, traffic-control, fleet management at scale
- Experience with Linux, Docker, and Kubernetes

**Bonus points:**
- Experience with modern networking stacks, including RDMA (RoCE, Infiniband, NVLink)
- Experience with high performance large scale ML systems (>100 GPUs)
- Experience with FFmpeg and multimedia processing

## Tech stack
**Must have:**
- Python
- Redis
- S3-compatible Storage
- Model serving (one of: PyTorch, vLLM, SGLang, Huggingface)
- Understanding of large-scale orchestration, deployment, scheduling (via Kubernetes or similar)

**Nice to have:**
- CUDA
- FFmpeg

## Compensation
The base pay range for this role is $187,500 – $395,000 per year.

## Similar roles

- [Software Engineer, ML Platform](https://hotfix.jobs/jobs/a9f2a8c2-33a8-4d0d-91c4-60f6a5135e7d) - Luma AI - Palo Alto, CA - $188k – $395k/yr
- [Research Scientist / Engineer – Training Infrastructure](https://hotfix.jobs/jobs/df564675-5da1-475c-a6b4-e81cc62da4eb) - Luma AI - Palo Alto, CA - $188k – $395k/yr
- [Software Engineer, Machine Learning Platform](https://hotfix.jobs/jobs/8227a48c-d3ba-4a85-afb5-a934accf63b3) - Chime - San Francisco, CA - $187k – $259k/yr
- [Agentic AI Engineer](https://hotfix.jobs/jobs/d74d66cb-3089-4707-b593-9f9e90c11b05) - EigenLayer - Remote - $187k – $253k/yr
- [Software Engineer - Behavior Capabilities](https://hotfix.jobs/jobs/07184ed0-bd8d-46fe-92cd-a18258309fb4) - Zoox - Foster City, CA - $189k – $270k/yr

**Apply:** https://hotfix.jobs/jobs/c31a9085-9d3a-4515-b43b-42516d45d8f1
**Canonical:** https://hotfix.jobs/jobs/c31a9085-9d3a-4515-b43b-42516d45d8f1