# Staff + Senior Software Engineer, Cloud Inference Launch Engineering

**Company:** [Anthropic](https://hotfix.jobs/companies/anthropic)
**Location:** San Francisco, CA
**Role:** ML Engineering
**Salary:** $320k – $485k/yr
**Experience:** 7+ years
**Skills:** Distributed Systems, Kubernetes, Infrastructure As Code, Python, Rust, AWS, GCP, Azure, CI/CD, Llm Inference, Load Balancing, Observability
**Posted:** 2026-07-14

> Build and own validation pipelines, CI/CD infrastructure, and platform integrations to launch frontier models and inference features reliably across AWS, GCP, and Azure. Requires strong large-scale distributed systems experience and track record improving release velocity.

## Job Description

## Key Responsibilities
- Be on the critical path for frontier model launches, bringing up inference for new model architectures and shipping them to cloud platforms in lockstep with our first-party platform.
- Work with the core inference team to bring new inference features (e.g. structured sampling, prompt caching) to cloud platforms, owning the platform-specific integration that gets them to production.
- Identify and dive deep on the gaps that make inference behave differently across first-party and CSPs (config drift, observability, deployment patterns, hard cross-platform bugs) and fix them at the source.
- Design, build, and own the CI/CD infrastructure for the inference server and load balancer across cloud platforms, with shadow traffic, performance baselines (throughput and latency), and correctness checks that catch regressions before production.
- Drive down merge-to-production cycle time by making validation faster, more parallel, and cost-effective enough to run on the same constrained accelerator pool that serves customers, without trading away reliability.
- Analyze observability data across providers to identify performance bottlenecks, cost anomalies, and regressions, and drive remediation based on real-world production workloads.

## Minimum Qualifications
- Strong interest in LLM serving (prior inference or ML experience not required).
- Significant software engineering experience with a strong background in high-performance, large-scale distributed systems serving millions of users.
- Track record of building automation or test infrastructure that measurably improved release velocity or reliability.
- Experience building or operating services on at least one major cloud platform (AWS, GCP, or Azure), with exposure to Kubernetes, Infrastructure as Code, or container orchestration.
- Thrive in cross-functional collaboration with both internal teams and external partners.
- Fast learner who can quickly ramp up on new technologies, hardware platforms, and provider ecosystems.
- Highly autonomous and take ownership of problems end-to-end.

## Preferred Qualifications
- LLM inference optimization, batching, and caching strategies.
- Capacity-constrained scheduling or shared-resource test infrastructure.
- Solid understanding of multi-region deployments, request routing, load balancing, global traffic management.
- Working with CSP partner teams to scale infrastructure across multiple platforms, navigating differences in networking, security, privacy, and managed services.
- Proficiency in Python or Rust.

## Compensation and Benefits
- Annual compensation range: $320,000–$485,000 USD (total compensation).
- Minimum education: Bachelor’s degree or equivalent.
- Location-based hybrid policy: expect staff to be in office at least 25% of the time.
- Competitive compensation, optional equity donation matching, generous vacation and parental leave, flexible working hours.

## Similar roles

- [Staff Software Engineer, Inference](https://hotfix.jobs/jobs/711c31e7-ec43-46d4-871a-d9074979e024) - Anthropic - San Francisco, CA - $320k – $485k/yr
- [Staff Software Engineer, Machine Learning](https://hotfix.jobs/jobs/e3a84742-62fc-4b45-8319-b472f5324c3d) - Attentive - Remote - $320k – $360k/yr
- [Member of Technical Staff](https://hotfix.jobs/jobs/be3c4b5e-6e62-4d15-9f3c-64b1b9d7e48e) - xAI - Palo Alto, CA - $324k – $396k/yr
- [Member of Technical Staff](https://hotfix.jobs/jobs/44e366a5-aab7-450a-8059-6b1900561147) - xAI - Palo Alto, CA - $324k – $396k/yr
- [Staff Research Engineer](https://hotfix.jobs/jobs/347c7f71-7f13-4e20-aeaf-742ee871d42c) - Decagon - San Francisco, CA - $325k – $425k/yr

**Apply:** https://hotfix.jobs/jobs/18b08a6a-5359-4836-b72e-c641fa718b04
**Canonical:** https://hotfix.jobs/jobs/18b08a6a-5359-4836-b72e-c641fa718b04