Skip to content
OpenAIOpenAISan Francisco, CA

Software Engineer, Inference - Multi Modal

Build and optimize high-performance inference infrastructure for OpenAI's multimodal models handling image, audio, and other non-text inputs at scale. Collaborate with research and product teams on low-latency production systems using GPU workloads and inference tooling.

295k – 555k/yr
On-siteML Engineering

About the role

Responsibilities

  • Design and implement inference infrastructure for large-scale multimodal models.
  • Optimize systems for high-throughput, low-latency delivery of image and audio inputs and outputs.
  • Enable experimental research workflows to transition into reliable production services.
  • Collaborate closely with researchers, infra teams, and product engineers to deploy state-of-the-art capabilities.
  • Contribute to system-level improvements including GPU utilization, tensor parallelism, and hardware abstraction layers.

Requirements

  • Experience building and scaling inference systems for LLMs or multimodal models.
  • Worked with GPU-based ML workloads and understand the performance dynamics of large models, especially with complex data like images or audio.
  • Enjoy experimental, fast-evolving work and collaborating closely with research.
  • Comfortable dealing with systems that span networking, distributed compute, and high-throughput data handling.
  • Familiarity with inference tooling like vLLM, TensorRT-LLM, or custom model parallel systems.
  • Own problems end-to-end and excited to operate in ambiguous, fast-moving spaces.

Nice to Have

  • Experience working with image generation or audio synthesis models in production.
  • Exposure to distributed ML training or system-efficient model design.

Skills

Inference SystemsLLMsMultimodal ModelsGpu WorkloadsvLLMTensorrt-LlmTensor ParallelismDistributed ComputeNetworkingHigh-Throughput Data Handling

Similar roles

ML Engineering jobs
OpenAI

Agent Post-Training, API & Power Users

OpenAISan Francisco, CA

Improve agentic model capabilities, reliability, and product fit for power users and API developers through evals, training data, and post-training interventions.

295k – 445k/yr
On-siteML Engineering
OpenAI

Software Engineer, RL Training Infra

OpenAISan Francisco, CA

Build and maintain infrastructure for large-scale RL training runs of frontier OpenAI models. Debug across training, inference, and distributed systems while supporting research integrations.

295k – 445k/yr
HybridML Engineering
OpenAI

Software Engineer, ML Systems & Training Architecture

OpenAISan Francisco, CA

Hands-on senior software engineer focused on maintaining and improving ML training infrastructure, debugging training systems, and unblocking researchers on the robotics team.

295k – 380k/yr
On-site3+ YOEML Engineering
OpenAI

Software Engineer, Inference - Performance Optimization

OpenAISan Francisco, CA

Models inference performance across application, model, and fleet layers using microbenchmarks to build cost-to-serve estimates. Analyzes workloads end-to-end, enhances bottleneck detection tools, and collaborates on optimizations for latency, throughput, and cost.

295k – 555k/yr
On-siteML Engineering
OpenAI

Machine Learning Engineer, Distributed Data Systems

OpenAISan Francisco, CA

Designs and scales distributed data infrastructure for large-scale multimodal AI training and evaluation. Collaborates with researchers to build reliable, high-performance systems in a fast-paced environment.

295k – 445k/yr
HybridML Engineering