# Performance Engineer, Inference Systems

**Company:** [Anthropic](https://hotfix.jobs/companies/anthropic)
**Location:** San Francisco, CA, New York, NY, Seattle, WA
**Role:** DevOps / SRE
**Salary:** $350k – $850k/yr
**Skills:** Python, Profiling, Roofline Analysis, Latency Optimization, Throughput Optimization, Root Cause Analysis, SQL, pandas, Observability, Telemetry, Gpu Performance, Model Evaluation, Llm Inference, Autoscaling, Distributed Systems
**Posted:** 2026-05-20

> Performance engineer focused on cross-layer investigations of Anthropic's inference fleet for Claude, optimizing throughput, latency, reliability, and correctness while building observability and partnering with kernel and serving teams.

## Job Description

## Key Responsibilities
- Run cross-layer performance investigations across throughput, latency, and reliability, sizing the gap between actual fleet performance and theoretical rooflines, identifying root causes, and quantifying the value of closing them
- Own and improve the correctness evaluation pipeline that validates model output quality across hardware platforms, numerics, and serving configurations, and lead the investigation when it catches a regression
- Build the observability, dashboards, and modeling tools that make throughput, latency, cost, reliability, correctness, and their interactions legible across the stack
- Partner with kernel, serving, routing, autoscaling, and capacity teams to prioritize and land the highest-impact optimizations your analysis surfaces
- Ruthlessly stack-rank a large surface area of opportunities by impact and effort, and say no to the ones that don't make the cut

## Minimum Qualifications
- Hands-on performance engineering experience: profiling, roofline analysis, latency/throughput optimization, and root-cause investigation in complex production systems
- Proficiency in Python, with the ability to read, instrument, and contribute to large production codebases you didn’t write
- Solid data analysis skills (e.g. SQL, pandas, or similar) sufficient to turn raw telemetry into clear findings
- Ability to communicate quantitative results clearly in writing to influence priorities on teams you don't manage
- Genuine interest in correctness as an engineering discipline: numerics, evaluation design, regression detection

## Preferred Qualifications
- Experience with ML systems, especially training or inference infrastructure or general LLM serving stacks. Direct large-scale inference experience is a strong plus
- Familiarity with GPU/TPU/accelerator performance concepts (memory bandwidth, kernel overheads, quantization, collective communication)
- Experience with reliability engineering for high-throughput services: autoscaling, load balancing, request routing, tail latency
- Experience with model evaluation or numerical regression-detection pipelines
- Experience building observability or telemetry for distributed systems
- Comfortable having impact through influence and evidence rather than direct ownership

## Representative Projects
- Trace a 350ms latency gap on a new accelerator platform from end-to-end request timing down to a server scheduling overhead, quantify the win, and land the fix directly or with the owning team
- Redesign the correctness eval gate: determine which signals reliably catch real model-output regressions versus noise, and make it the trusted release criterion across hardware backends
- Build a FLOPs funnel that breaks down where compute actually goes across the fleet, exposing the gap between achieved throughput and kernel rooflines
- Root-cause a numerical divergence between two hardware platforms to a specific kernel change, and define the acceptance threshold going forward
- Model the latency–cost impact of changing batch-sizing and utilization targets, and turn the result into the signal the autoscaler uses in production

## Similar roles

- [Reliability Engineer, Supercomputing](https://hotfix.jobs/jobs/e9263e4b-66f2-4203-b8da-b88ba1855dfe) - Thinking Machines Lab - San Francisco, CA - $350k – $475k/yr
- [Network Engineer, Supercomputing](https://hotfix.jobs/jobs/2a952a8c-f34c-4fdd-a219-c8ac22399145) - Thinking Machines Lab - San Francisco, CA - $350k – $475k/yr
- [Site Reliability Engineer (SRE)](https://hotfix.jobs/jobs/893a897e-0421-40b8-a8ac-f61c0b30d5ff) - Thinking Machines Lab - San Francisco, CA - $350k – $475k/yr
- [Research Engineer, Infrastructure, Training Systems](https://hotfix.jobs/jobs/9cc8be27-572e-4c19-97d0-8a7ae54e41a8) - Thinking Machines Lab - San Francisco, CA - $350k – $475k/yr
- [Research Engineer, Infrastructure, RL Systems](https://hotfix.jobs/jobs/71d52d2b-ffa8-42f6-a7af-3a117bfd347b) - Thinking Machines Lab - San Francisco, CA - $350k – $475k/yr

**Apply:** https://hotfix.jobs/jobs/c4e8b3e8-25ad-4a83-b11d-bebf411d63f1
**Canonical:** https://hotfix.jobs/jobs/c4e8b3e8-25ad-4a83-b11d-bebf411d63f1