Skip to content
ZooxZooxFoster City, CA

AI Inference Engineer - Model Optimization & Deployment

Optimizes and deploys large-scale AI models (LLMs, VLMs) for real-time inference on power-constrained vehicle hardware. Requires expertise in quantization, TensorRT compilation, custom CUDA kernels, and production C++/Python for edge devices.

242k – 290k/yr
HybridML Engineering

About the role

Responsibilities

  • Optimize large-scale models (LLMs, VLMs) using advanced quantization (PTQ, QAT), mixed-precision inference workflows, and parameter-efficient fine-tuning (LoRA, QLoRA).
  • Architect and implement model conversion and compilation pipelines using TensorRT and TensorRT-LLM for edge deployment.
  • Perform rigorous parity checking, accuracy recovery, and latency benchmarking between PyTorch frameworks and compiled edge binaries.
  • Write and optimize custom CUDA kernels and TensorRT Plugins to maximize memory bandwidth and minimize latency on AI accelerators.
  • Write production-level, highly concurrent, and memory-safe C++ and Python code for real-time inference on vehicle SOCs.

Qualifications

  • Deep expertise in model quantization (PTQ, QAT) and mixed-precision inference workflows (INT8, FP8, INT4, BF16/FP16).
  • Proven experience optimizing large-scale models (LLMs, VLMs, or VLAs) utilizing KV-cache optimization (e.g., PagedAttention), Speculative Decoding, and Efficient Attention mechanisms (FlashAttention, Linear Attention).
  • Extensive experience with model conversion/compilation pipelines (TensorRT, TensorRT-LLM) and performing rigorous parity/latency benchmarking.
  • Proficiency in low-level programming for AI accelerators, specifically writing and optimizing custom CUDA kernels and TensorRT Plugins.
  • Production-level C++ (14/17/20) and Python programming skills, with experience writing concurrent, memory-safe, real-time inference code for edge devices.

Bonus Qualifications

  • Experience with distributed training pipelines and model/tensor parallelism (PyTorch Distributed, Ray, DeepSpeed, Megatron-LM) and runtime efficiency optimization for GPU clusters.
  • Familiarity with autonomous driving perception stacks (temporal 3D object detection, BEV, 3D Occupancy Networks) and processing multi-modal sensor streams (Vision, LiDAR, Radar).
  • Understanding of end-to-end autonomous driving paradigms (VLA models, closed-loop simulation validation).

Skills

TensorRTTensorrt-LlmCUDAPyTorchC++PythonPtqQatLoraQloraFlashattentionPagedattentionDeepspeedRay

Similar roles

ML Engineering jobs
Abridge

Clinician Scientist

AbridgeSan Francisco, CA

Combines clinical expertise (MD/PA/DO) with AI to develop and refine tools for clinical documentation, decision support, and risk adjustment. Defines quality standards, optimizes prompts, evaluates outputs, and collaborates with engineering teams to enhance clinician efficiency.

236k – 277k/yr
HybridML Engineering
OpenAI

Researcher, Alignment Oversight

OpenAISan Francisco, CA

Designs and runs experiments to improve oversight of increasingly capable AI models, including model training, evaluation, and deployment of practical systems. Analyzes failures and develops techniques to train more aligned models using oversight signals.

250k – 445k/yr
HybridML Engineering
Luma AI

Research Scientist / Engineer — Multimodal Agent

Luma AIPalo Alto, CA

Builds and trains large-scale multimodal agentic models involving reasoning, planning, coding, and tool calling. Requires strong ML foundations, PyTorch expertise, and experience with distributed training on massive datasets.

250k – 450k/yr
HybridML Engineering
Variance

Research Engineer, Evals

VarianceSan Francisco, CA

Build benchmarks, datasets, and evaluation systems to measure and improve AI model quality for fraud, identity, and risk judgment tasks. Collaborate across research, engineering, and product to drive rigorous experimentation and iteration in high-stakes environments.

250k – 400k/yr
On-siteML Engineering
Variance

Research Engineer, Judgment Systems

VarianceSan Francisco, CA

Research Engineer designs evaluations, studies model failures, and builds research loops to improve AI agents for high-stakes fraud detection and judgment tasks. Requires ML training experience, experimental rigor, and strong engineering skills in adversarial environments.

250k – 400k/yr
On-siteML Engineering