Skip to content
OpenAIOpenAISan Francisco, CA

TL, Research Inference

Leads development of high-performance inference systems for large-scale AI models, optimizing execution paths, distributed GPU inference, and operators. Partners with research teams to enable efficient exploration of new architectures grounded in real scalability constraints.

380k – 555k
On-siteML Engineering

About the role

Responsibilities

  • Design and build high-performance inference runtimes for large-scale AI models, with a focus on efficiency, reliability, and scalability.
  • Own and optimize core execution paths, including model execution, memory management, batching, and scheduling.
  • Develop and improve distributed inference across multiple GPUs, including parallelism strategies, communication patterns, and runtime coordination.
  • Implement and optimize inference-critical operators and kernels informed by real-world workloads.
  • Partner closely with research teams to ensure new model architectures are supported accurately and efficiently in inference systems.
  • Diagnose and resolve performance bottlenecks through profiling, benchmarking, and low-level debugging.
  • Contribute to observability, correctness, and reliability of large-scale AI systems.

Requirements

  • Experience building production inference systems, not just training or running models.
  • Comfortable with GPU-centric performance engineering, including memory behavior and latency/throughput tradeoffs.
  • Worked on multi-GPU or distributed systems involving batching, scheduling, or runtime coordination.
  • Can reason end-to-end about inference pipelines, from request handling through execution and output streaming.
  • Able to understand research ideas and implement them within real system and performance constraints.
  • Enjoy solving hard, ambiguous systems problems that only emerge at scale.
  • Prefer hands-on technical ownership and execution over abstract design work.

Skills

Gpu ProgrammingDistributed SystemsInference OptimizationPyTorchTensorRTCUDAMulti-GpuBatchingSchedulingMemory ManagementProfilingBenchmarking

Similar roles

ML Engineering jobs
OpenAI

Research Engineer, Privacy

OpenAISan Francisco, CA

Research Engineer on OpenAI's Privacy team designing and prototyping privacy-preserving ML algorithms like differential privacy and federated learning at scale. Requires hands-on PETs experience, fluency in PyTorch/JAX, and a track record implementing or publishing novel privacy work.

380k – 445k
HybridML Engineering
Thinking Machines Lab

Research Infrastructure Engineer

Thinking Machines LabSan Francisco, CA

Build and operate research infrastructure like evaluation frameworks, RL training systems, and experiment tracking platforms. Partner directly with ML researchers to identify bottlenecks, ensure high adoption of tools, and accelerate research velocity.

350k – 475k
On-siteML Engineering
Anthropic

Research Engineer, Safeguards Labs

AnthropicSan Francisco, CA +1

Research engineer on the Safeguards Labs team building and evaluating novel safety methods to detect misuse, strengthen model safeguards, and reduce real-world harm from Claude.

350k – 850k
HybridML Engineering
Anthropic

Research Engineer, Knowledge Team

AnthropicSan Francisco, CA +2

Designs new information architectures for LLMs to interact with external data sources, implements finetuning/RL training, builds evaluation sets, and develops agentic search capabilities. Requires strong Python/ML skills and LLM experience.

350k – 850k
HybridML Engineering
Thinking Machines Lab

Research, Vision Expertise

Thinking Machines LabSan Francisco, CA

Conducts research on visual perception, multimodal learning, and large-scale AI model training. Designs architectures, builds datasets and evaluations, and collaborates on frontier models. Requires ML expertise, Python proficiency, and experimental rigor.

350k – 475k
On-siteML Engineering