Skip to content
Luma AILuma AIPalo Alto, CA

Software Engineer, Inference

Develops and optimizes inference engines for multimodal AI models, integrating new architectures, building scheduling systems, and managing large-scale GPU deployments. Requires strong Python, model serving frameworks like PyTorch/vLLM, and Kubernetes expertise.

188k – 395k/yr
HybridML Engineering

About the role

Role & Responsibilities

  • Ship new model architectures by integrating them into our inference engine
  • Collaborate closely across research, engineering and infrastructure to streamline and optimize model efficiency and deployments
  • Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows
  • Automate, test and maintain our inference services to ensure maximum uptime and reliability
  • Optimize deployment workflows to scale across thousands of machines
  • Manage and optimize our inference workloads across different clusters & hardware providers
  • Build sophisticated scheduling systems to optimally leverage our expensive GPU resources while meeting internal SLOs
  • Build and maintain CI/CD pipelines for processing/optimizing model checkpoints, platform components, and SDKs for internal teams to integrate into our products/internal tooling

Background

Must have:

  • Strong Python and system architecture skills
  • Experience with model deployment using PyTorch, Huggingface, vLLM, SGLang, tensorRT-LLM, or similar
  • Experience with queues, scheduling, traffic-control, fleet management at scale
  • Experience with Linux, Docker, and Kubernetes

Bonus points:

  • Experience with modern networking stacks, including RDMA (RoCE, Infiniband, NVLink)
  • Experience with high performance large scale ML systems (>100 GPUs)
  • Experience with FFmpeg and multimedia processing

Tech stack

Must have:

  • Python
  • Redis
  • S3-compatible Storage
  • Model serving (one of: PyTorch, vLLM, SGLang, Huggingface)
  • Understanding of large-scale orchestration, deployment, scheduling (via Kubernetes or similar)

Nice to have:

  • CUDA
  • FFmpeg

Compensation

The base pay range for this role is $187,500 – $395,000 per year.

Skills

PythonPyTorchHuggingfacevLLMSglangTensorrt-LlmKubernetesDockerLinuxRedis

Similar roles

ML Engineering jobs
Luma AI

Software Engineer, ML Platform

Luma AIPalo Alto, CA

Builds foundational ML platform infrastructure including model serving pipelines, GPU scheduling systems, and CI/CD for large-scale multimodal AI models. Requires 5+ years in distributed systems with expertise in Python, Kubernetes, and AWS.

188k – 395k/yr
Hybrid5+ YOEML Engineering
Luma AI

Research Scientist / Engineer – Training Infrastructure

Luma AIPalo Alto, CA

Builds and optimizes distributed training infrastructure for large-scale multimodal AI models across thousands of GPUs. Requires deep expertise in PyTorch, CUDA, parallelization techniques, and GPU clusters.

188k – 395k/yr
HybridML Engineering
Chime

Software Engineer, Machine Learning Platform

ChimeSan Francisco, CA

Build and operate Chime's ML platform on AWS, including distributed training systems, feature stores, data pipelines, and CI/CD tooling. Partner with ML teams to improve reliability, observability, and developer experience for production models.

187k – 259k/yr
Hybrid5+ YOEML Engineering
EigenLayer

Agentic AI Engineer

EigenLayerSeattle, WA +1

Builds production-ready agentic AI systems including runtimes, orchestration, reliability, observability, and integrations with LLMs/APIs. Requires strong backend experience, shipped agent/LLM systems, and production reliability expertise.

187k – 253k/yr
RemoteML Engineering
Zoox

Software Engineer - Behavior Capabilities

ZooxFoster City, CA

As a Software Engineer on the Behavior Capabilities team, you will develop and implement algorithmic advancements to expand the robot's driving abilities in complex scenarios, focusing on improving trip progress and vehicle uptime.

189k – 270k/yr
Hybrid4+ YOEML Engineering