# Software Engineer, Inference – AMD GPU Enablement

**Company:** [OpenAI](https://hotfix.jobs/companies/openai)
**Location:** San Francisco, CA
**Role:** ML Engineering
**Salary:** $295k – $555k/yr
**Skills:** Hip, CUDA, Triton, vLLM, Nccl, Rccl, Rocm, Distributed Inference, Gpu Kernels, Model Parallelism
**Posted:** 2025-10-08

> Develops and optimizes OpenAI's inference infrastructure for AMD GPUs, handling low-level kernel performance, distributed execution, and integration with serving frameworks like vLLM and Triton. Requires expertise in GPU programming with HIP/CUDA and distributed systems scaling.

## Job Description

## Responsibilities
- Own bring-up, correctness and performance of the OpenAI inference stack on AMD hardware.
- Integrate internal model-serving infrastructure (e.g., vLLM, Triton) into a variety of GPU-backed systems.
- Debug and optimize distributed inference workloads across memory, network, and compute layers.
- Validate correctness, performance, and scalability of model execution on large GPU clusters.
- Collaborate with partner teams to design and optimize high-performance GPU kernels for accelerators using HIP, Triton, or other performance-focused frameworks.
- Collaborate with partner teams to build, integrate and tune collective communication libraries (e.g., RCCL) used to parallelize model execution across many GPUs.

## Requirements
- Experience writing or porting GPU kernels using HIP, CUDA, or Triton, and care deeply about low-level performance.
- Familiar with communication libraries like NCCL/RCCL and understand their role in high-throughput model serving.
- Worked on distributed inference systems and comfortable scaling models across fleets of accelerators.
- Enjoy solving end-to-end performance challenges across hardware, system libraries, and orchestration layers.
- Excited to be part of a small, fast-moving team building new infrastructure from first principles.

## Nice to Have
- Contributions to open-source libraries like RCCL, Triton, or vLLM.
- Experience with GPU performance tools (Nsight, rocprof, perf) and memory/comms profiling.
- Prior experience deploying inference on other non-NVIDIA GPU environments.
- Knowledge of model/tensor parallelism, mixed precision, and serving 10B+ parameter models.

## Similar roles

- [Agent Post-Training, API & Power Users](https://hotfix.jobs/jobs/fe6ae516-bafa-4cc6-af78-f20aa90a9eb3) - OpenAI - San Francisco, CA - $295k – $445k/yr
- [Software Engineer, RL Training Infra](https://hotfix.jobs/jobs/5c9ca321-9012-4a35-9c66-fbd05cddb52e) - OpenAI - San Francisco, CA - $295k – $445k/yr
- [Software Engineer, ML Systems & Training Architecture](https://hotfix.jobs/jobs/09ff26d4-974c-471b-9d70-82c93f3c6aab) - OpenAI - San Francisco, CA - $295k – $380k/yr
- [Software Engineer, Inference - Performance Optimization](https://hotfix.jobs/jobs/0f2ff1a5-79fe-4fe9-ad20-d84aec238f5e) - OpenAI - San Francisco, CA - $295k – $555k/yr
- [Machine Learning Engineer, Distributed Data Systems](https://hotfix.jobs/jobs/deb54433-eb08-4c3e-adee-afee842e3253) - OpenAI - San Francisco, CA - $295k – $445k/yr

**Apply:** https://hotfix.jobs/jobs/55c79b8e-5508-43cd-9f34-af4fa918e81a
**Canonical:** https://hotfix.jobs/jobs/55c79b8e-5508-43cd-9f34-af4fa918e81a