Skip to content
OpenAIOpenAISan Francisco, CA

Software Engineer, Inference – AMD GPU Enablement

Develops and optimizes OpenAI's inference infrastructure for AMD GPUs, handling low-level kernel performance, distributed execution, and integration with serving frameworks like vLLM and Triton. Requires expertise in GPU programming with HIP/CUDA and distributed systems scaling.

295k – 555k/yr
On-siteML Engineering

About the role

Responsibilities

  • Own bring-up, correctness and performance of the OpenAI inference stack on AMD hardware.
  • Integrate internal model-serving infrastructure (e.g., vLLM, Triton) into a variety of GPU-backed systems.
  • Debug and optimize distributed inference workloads across memory, network, and compute layers.
  • Validate correctness, performance, and scalability of model execution on large GPU clusters.
  • Collaborate with partner teams to design and optimize high-performance GPU kernels for accelerators using HIP, Triton, or other performance-focused frameworks.
  • Collaborate with partner teams to build, integrate and tune collective communication libraries (e.g., RCCL) used to parallelize model execution across many GPUs.

Requirements

  • Experience writing or porting GPU kernels using HIP, CUDA, or Triton, and care deeply about low-level performance.
  • Familiar with communication libraries like NCCL/RCCL and understand their role in high-throughput model serving.
  • Worked on distributed inference systems and comfortable scaling models across fleets of accelerators.
  • Enjoy solving end-to-end performance challenges across hardware, system libraries, and orchestration layers.
  • Excited to be part of a small, fast-moving team building new infrastructure from first principles.

Nice to Have

  • Contributions to open-source libraries like RCCL, Triton, or vLLM.
  • Experience with GPU performance tools (Nsight, rocprof, perf) and memory/comms profiling.
  • Prior experience deploying inference on other non-NVIDIA GPU environments.
  • Knowledge of model/tensor parallelism, mixed precision, and serving 10B+ parameter models.

Skills

HipCUDATritonvLLMNcclRcclRocmDistributed InferenceGpu KernelsModel Parallelism

Similar roles

ML Engineering jobs
OpenAI

Agent Post-Training, API & Power Users

OpenAISan Francisco, CA

Improve agentic model capabilities, reliability, and product fit for power users and API developers through evals, training data, and post-training interventions.

295k – 445k/yr
On-siteML Engineering
OpenAI

Software Engineer, RL Training Infra

OpenAISan Francisco, CA

Build and maintain infrastructure for large-scale RL training runs of frontier OpenAI models. Debug across training, inference, and distributed systems while supporting research integrations.

295k – 445k/yr
HybridML Engineering
OpenAI

Software Engineer, ML Systems & Training Architecture

OpenAISan Francisco, CA

Hands-on senior software engineer focused on maintaining and improving ML training infrastructure, debugging training systems, and unblocking researchers on the robotics team.

295k – 380k/yr
On-site3+ YOEML Engineering
OpenAI

Software Engineer, Inference - Performance Optimization

OpenAISan Francisco, CA

Models inference performance across application, model, and fleet layers using microbenchmarks to build cost-to-serve estimates. Analyzes workloads end-to-end, enhances bottleneck detection tools, and collaborates on optimizations for latency, throughput, and cost.

295k – 555k/yr
On-siteML Engineering
OpenAI

Machine Learning Engineer, Distributed Data Systems

OpenAISan Francisco, CA

Designs and scales distributed data infrastructure for large-scale multimodal AI training and evaluation. Collaborates with researchers to build reliable, high-performance systems in a fast-paced environment.

295k – 445k/yr
HybridML Engineering