# Software Engineer, Model Inference

**Company:** [OpenAI](https://hotfix.jobs/companies/openai)
**Location:** San Francisco, CA
**Role:** ML Engineering
**Salary:** $295k – $555k/yr
**Experience:** 5+ years
**Skills:** PyTorch, Nvidia Gpus, CUDA, Nccl, InfiniBand, Mpi, Nvlink, Azure, Distributed Systems, Ml Inference
**Posted:** 2025-02-06

> Optimizes large AI models for high-volume, low-latency production and research environments. Collaborates with researchers and engineers on inference stack performance, requiring 5+ years experience with PyTorch, GPUs, CUDA, and distributed systems.

## Job Description

## Responsibilities
- Work alongside machine learning researchers, engineers, and product managers to bring latest technologies into production.
- Enable advanced research through engineering.
- Introduce new techniques, tools, and architecture to improve performance, latency, throughput, and efficiency of model inference stack.
- Build tools for visibility into bottlenecks and instability, then design and implement solutions.
- Optimize code and Azure VMs to maximize GPU utilization.

## Requirements
- Understanding of modern ML architectures and optimization for inference.
- Own problems end-to-end and learn as needed.
- At least 5 years of professional software engineering experience.
- Familiarity with PyTorch, NVIDIA GPUs, NCCL, CUDA, HPC technologies (InfiniBand, MPI, NVLink).
- Experience architecting, building, observing, and debugging production distributed systems (bonus for performance-critical).
- Experience rebuilding/refactoring production systems at scale.
- Self-directed, humble, eager to help team.

## Nice-to-Haves
- Performance-critical distributed systems experience.

## Similar roles

- [Agent Post-Training, API & Power Users](https://hotfix.jobs/jobs/fe6ae516-bafa-4cc6-af78-f20aa90a9eb3) - OpenAI - San Francisco, CA - $295k – $445k/yr
- [Software Engineer, RL Training Infra](https://hotfix.jobs/jobs/5c9ca321-9012-4a35-9c66-fbd05cddb52e) - OpenAI - San Francisco, CA - $295k – $445k/yr
- [Software Engineer, ML Systems & Training Architecture](https://hotfix.jobs/jobs/09ff26d4-974c-471b-9d70-82c93f3c6aab) - OpenAI - San Francisco, CA - $295k – $380k/yr
- [Software Engineer, Inference - Performance Optimization](https://hotfix.jobs/jobs/0f2ff1a5-79fe-4fe9-ad20-d84aec238f5e) - OpenAI - San Francisco, CA - $295k – $555k/yr
- [Machine Learning Engineer, Distributed Data Systems](https://hotfix.jobs/jobs/deb54433-eb08-4c3e-adee-afee842e3253) - OpenAI - San Francisco, CA - $295k – $445k/yr

**Apply:** https://hotfix.jobs/jobs/8a5c9440-89c0-45e4-805c-3d366e2cf647
**Canonical:** https://hotfix.jobs/jobs/8a5c9440-89c0-45e4-805c-3d366e2cf647