Skip to content
OpenAIOpenAISan Francisco, CA

Software Engineer, Inference - Performance Optimization

Models inference performance across application, model, and fleet layers using microbenchmarks to build cost-to-serve estimates. Analyzes workloads end-to-end, enhances bottleneck detection tools, and collaborates on optimizations for latency, throughput, and cost.

295k – 555k/yr
On-siteML Engineering

About the role

Responsibilities

  • Build and refine performance models that translate microbenchmark results into cost-to-serve estimates.
  • Analyze inference workloads end to end across applications, models, and fleet infrastructure.
  • Enhance tooling to identify bottlenecks across layers for latency and throughput.
  • Partner with other teams to turn performance insights into concrete improvements and project how future changes affect inference.

Requirements

  • Enjoy reasoning from first principles about distributed systems, model inference, and hardware efficiency.
  • Comfortable working across abstraction layers, from application behavior to kernels, accelerators, networking, and fleet scheduling.
  • Deep expertise with performance profiling, benchmarking, analysis, and optimization.
  • Enjoy collaborating with engineering and research teams to improve real production systems.

Skills

Performance ModelingProfilingBenchmarkingDistributed SystemsModel InferenceHardware OptimizationKernelsAcceleratorsNetworkingFleet Scheduling

Similar roles

ML Engineering jobs
OpenAI

Agent Post-Training, API & Power Users

OpenAISan Francisco, CA

Improve agentic model capabilities, reliability, and product fit for power users and API developers through evals, training data, and post-training interventions.

295k – 445k/yr
On-siteML Engineering
OpenAI

Software Engineer, RL Training Infra

OpenAISan Francisco, CA

Build and maintain infrastructure for large-scale RL training runs of frontier OpenAI models. Debug across training, inference, and distributed systems while supporting research integrations.

295k – 445k/yr
HybridML Engineering
OpenAI

Software Engineer, ML Systems & Training Architecture

OpenAISan Francisco, CA

Hands-on senior software engineer focused on maintaining and improving ML training infrastructure, debugging training systems, and unblocking researchers on the robotics team.

295k – 380k/yr
On-site3+ YOEML Engineering
OpenAI

Machine Learning Engineer, Distributed Data Systems

OpenAISan Francisco, CA

Designs and scales distributed data infrastructure for large-scale multimodal AI training and evaluation. Collaborates with researchers to build reliable, high-performance systems in a fast-paced environment.

295k – 445k/yr
HybridML Engineering
OpenAI

Software Engineer, Inference – AMD GPU Enablement

OpenAISan Francisco, CA

Develops and optimizes OpenAI's inference infrastructure for AMD GPUs, handling low-level kernel performance, distributed execution, and integration with serving frameworks like vLLM and Triton. Requires expertise in GPU programming with HIP/CUDA and distributed systems scaling.

295k – 555k/yr
On-siteML Engineering