Staff+ Software Engineer, ML Sampling Path

Responsibilities

• Design, build, and operate backend systems that process every token on Claude’s generation path, including streaming contracts with APIs and inference engines.
• Own latency and reliability end to end by defining and maintaining SLOs and error budgets for added latency, time-to-first-token, and availability.
• Lead incident response and postmortem follow-through.
• Ship changes to critical hot-path systems through canaried and gradual rollouts, latency and error-budget gating, and rapid rollbacks.
• Optimize per-token performance, tail latency, and cost as traffic, models, and checks per request grow.
• Set technical direction through design reviews and latency, reliability, and cost tradeoff decisions with inference and research teams.
• Mentor engineers and raise operational standards across the safeguards organization.

Requirements

• Experience designing, building, and operating high-QPS systems at global scale in production.
• Experience with incident response, outages, and postmortem-driven remediation.
• Strong foundation in distributed systems, including replication, consistency tradeoffs, failure modes, and SLO management under load.
• Ability to design graceful degradation for slow dependencies, dropped streams, and partially rolled-out deployments.
• Experience shipping broad changes to mission-critical systems, such as database migrations, interface changes, or rewrites.
• 8+ years of industry software engineering experience.
• Bachelor’s degree or equivalent combination of education, training, and/or experience in a relevant field.

Nice-to-haves

• Familiarity with LLM inference systems.
• Familiarity with transformer-based models.

Compensation

• Annual salary: $320,000–$485,000 USD
• Competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, and flexible working hours.