Skip to content
PerplexityPerplexitySan Francisco, CA

AI Inference Engineer

Develops APIs and optimizes large-scale ML model inference using Python, Rust, C++, PyTorch, and CUDA. Benchmarks performance, improves reliability, and implements LLM optimizations on GPU architectures.

220k – 405k/yr
On-siteML Engineering

About the role

Responsibilities

  • Develop APIs for AI inference that will be used by both internal and external customers
  • Benchmark and address bottlenecks throughout our inference stack
  • Improve the reliability and observability of our systems and respond to system outages
  • Explore novel research and implement LLM inference optimizations

Qualifications

  • Experience with ML systems and deep learning frameworks (e.g. PyTorch, TensorFlow, ONNX)
  • Familiarity with common LLM architectures and inference optimization techniques (e.g. continuous batching, quantization, etc.)
  • Understanding of GPU architectures or experience with GPU kernel programming using CUDA

Skills

PythonRustC++PyTorchTritonCUDAKubernetesTensorFlowOnnx

Similar roles

ML Engineering jobs
Perplexity

AI Engineer, Applied ML

PerplexitySan Francisco, CA +2

Designs, builds, and iterates on AI/ML models for personalization, query understanding, and content discovery. Requires 5+ years in ML, deep learning expertise (PyTorch/TensorFlow/JAX), Python, and full ML lifecycle ownership.

220k – 405k/yr
On-site5+ YOEML Engineering
Abridge

Machine Learning Infrastructure Engineer- Model Inference

AbridgeSan Francisco, CA

Builds and optimizes scalable ML inference infrastructure using Kubernetes and GPU resources to deploy production AI models with low latency. Collaborates with ML research and product teams on model serving, orchestration, and compute efficiency.

221k – 260k/yr
HybridML Engineering
Scale AI

ML Systems Engineer, Robotics

Scale AISan Francisco, CA

Designs and builds scalable platforms for serving foundation models in robotics and autonomous vehicles. Requires 4+ years in ML infrastructure, backend systems, GPU optimization, and cloud technologies.

218k – 273k/yr
On-site4+ YOEML Engineering
Scale AI

ML Research Engineer, ML Systems

Scale AISan Francisco, CA +2

Builds and optimizes distributed frameworks for LLM training and inference on Scale's RLXF platform. Collaborates with ML teams to accelerate research, requiring expertise in PyTorch, CUDA, transformers, and large-scale distributed systems.

218k – 273k/yr
On-siteML Engineering
Scale AI

Machine Learning Research Engineer, Agent Data Foundation - Enterprise GenAI

Scale AISan Francisco, CA +2

Develops synthetic data pipelines, production trace agents, and automated agent-building frameworks for enterprise GenAI. Requires 3+ years LLM production experience, top conference publications, and advanced CS degree.

218k – 273k/yr
Hybrid3+ YOEML Engineering