Software Engineer, AI Reliability
Software Engineer focused on AI reliability engineering, improving robustness of Claude's serving infrastructure across SDK to accelerators. Partners cross-team on SLOs, monitoring, high-availability systems, incident response, and safeguard models.
About the job
Responsibilities
- Develop appropriate Service Level Objectives for large language model serving systems, balancing availability and latency with development velocity.
- Design and implement monitoring and observability systems across the token path.
- Assist in the design and implementation of high-availability serving infrastructure across multiple regions and cloud providers.
- Lead incident response for critical AI services, ensuring rapid recovery, thorough incident reviews, and systematic improvements.
- Support the reliability of safeguard model serving—critical for both site reliability and Anthropic's safety commitments.
You may be a good fit if you
- Have strong distributed systems, infrastructure, or reliability backgrounds—we're looking for reliability-minded software engineers and SREs.
- Are curious and brave—comfortable jumping into unfamiliar systems during an incident and helping drive resolution even when you don't have deep expertise yet.
- Think holistically about how systems compose and where the seams are.
- Can build lasting relationships across teams—our engagement model depends on being welcomed as teammates, not outsiders with opinions.
- Care about users and feel ownership over outcomes, even for systems you don't own.
- Have excellent communication and collaboration skills—you'll be partnering across the entire company.
- Bring diverse experience—the team's strength comes from people who've built product stacks, scaled databases, run massive distributed systems, and everything in between.
Strong candidates may also
- Have been an SRE, Production Engineer, or in similar reliability-focused roles on large scale systems.
- Have experience operating large-scale model serving or training infrastructure (>1000 GPUs).
- Have experience with one or more ML hardware accelerators (GPUs, TPUs, Trainium).
- Understand ML-specific networking optimizations like RDMA and InfiniBand.
- Have expertise in AI-specific observability tools and frameworks.
- Have experience with chaos engineering and systematic resilience testing.
- Have contributed to open-source infrastructure or ML tooling.
Logistics
Annual Salary: $325,000—$485,000 USD
Education requirements: We require at least a Bachelor's degree in a related field or equivalent experience.
Location-based hybrid policy: Currently, we expect all staff to be in one of our offices at least 25% of the time. However, some roles may require more time in our offices.
Skills
Distributed Systems, SRE, Observability, Monitoring, High-Availability Infrastructure, Incident Response, Gpus, Tpus, Rdma, Chaos Engineering
Similar jobs
ML Engineering jobs
OpenAI
San Francisco, CA
Develop multimodal perception and authentication systems combining visual, audio, and other sensor signals for real-world AI products. The role requires machine learning expertise, practical research-to-system experience, and proficiency in Python and PyTorch with comfort in C++.
Thinking Machines Lab
San Francisco, CA
Build and operate the engineering systems that support post-training research, including reinforcement learning infrastructure, sandboxed execution, data pipelines, and agent scaffolding. The role requires strong Python and systems engineering skills, project ownership, and a relevant bachelor’s degree or equivalent experience.
Thinking Machines Lab
San Francisco, CA
Operates and improves the infrastructure powering large-scale post-training and reinforcement learning runs, partnering with researchers to debug failures, improve reliability, and automate recovery. Requires 4+ years operating distributed production systems and strong Python, Go, or C++ skills.
Thinking Machines Lab
San Francisco, CA
Research-focused engineer advancing agentic model capabilities across synthetic data, task environments, evaluations, training, and usability improvements. Requires strong Python engineering, deep learning framework experience, scalable distributed training skills, and scientific experimentation ability.
Thinking Machines Lab
San Francisco, CA
Researcher focused on scaling reinforcement learning for frontier models, with ownership spanning asynchronous RL algorithms, inference and distributed training systems, and large-scale empirical studies. Requires strong Python and deep learning experience, scalable systems debugging, and rigorous research judgment.