Staff Software Engineer, Node Infra
Own technical strategy and roadmap for node lifecycle management, health automation, and scaling AI clusters across clouds and accelerators. Requires deep distributed systems expertise, ML accelerator experience, and 12+ years leading complex multi-team infrastructure initiatives.
About the job
Key responsibilities
- Own the technical strategy and roadmap for node lifecycle management - ingestion, bring-up, health checking, and automated repair
- Drive cross-team initiatives to build and scale AI clusters across multiple clouds and accelerator families
- Design and operate the systems that detect, isolate, and remediate unhealthy hardware automatically, driving up fleet MTBI and minimizing stranded capacity
- Define infrastructure architecture, ensuring the hardest problems get solved - whether by you directly or by working through others
- Work closely with cloud providers and internal research/inference/product teams to shape long-term compute, data, and infrastructure strategy
- Establish and evolve operational excellence practices (incident response, postmortem culture, on-call)
- Support the growth of engineers around you through technical mentorship and coaching
Minimum qualifications
- Deep expertise in distributed systems, reliability, and cloud platforms (e.g., Kubernetes, IaC, AWS/GCP/Azure)
- Strong proficiency in at least one systems language (e.g., Rust, Go, or Python), IaC proficiency with Terraform
- Hands-on experience with machine learning accelerators (GPUs, TPUs, or Trainium)
- Track record of leading complex, multi-quarter technical initiatives that span multiple teams or systems
- Ability to build alignment across senior stakeholders and communicate effectively at all levels
Preferred qualifications
- 12+ years of software engineering experience, including time as a technical lead setting direction for a team
- Experience managing large scale compute infrastructure at hyperscale (10K+ nodes), including capacity management and efficiency
- Depth in one or more of: Kubernetes internals (scheduler, autoscaler, kubelet, Karpenter), cluster orchestration systems (Mesos, Borg-like), or node provisioning pipelines
- Low-level systems experience: kernel, virtualization, device drivers, firmware, or hardware health/diagnostics daemons
- Familiarity with high-performance networking (EFA, RDMA, InfiniBand) for distributed ML workloads
- Demonstrated ownership of production reliability for high-throughput, latency-sensitive systems
- Contributions to relevant open-source projects (Kubernetes, Linux kernel, container runtimes, etc.)
- Skill in quickly understanding systems design tradeoffs and keeping track of rapidly evolving software systems
Compensation
- Annual Salary: $405,000—$485,000 USD
Skills
Kubernetes, Terraform, AWS, GCP, Azure, Rust, Go, Python, Gpus, Tpus, Distributed Systems, Iac
Similar jobs
DevOps / SRE jobs
Anthropic
San Francisco, CABuild and operate portable infrastructure that enables Claude to run reliably across multiple cloud providers and accelerator platforms. The role requires 8+ years of distributed-systems experience, multi-cloud architecture expertise, production programming, Kubernetes, and Infrastructure as Code proficiency.
Anthropic
London, United Kingdom
Build and operate foundational observability infrastructure spanning telemetry pipelines, profiling, tracing, and diagnostic tooling across large-scale compute clusters. The role requires deep systems-level experience and 10+ years of relevant industry experience.
Anthropic
London, United Kingdom
Leads reliability engineering for critical AI serving systems, spanning SLOs, observability, high availability, and incident response. Requires strong distributed-systems or infrastructure experience, with model-serving, accelerator, networking, and resilience-testing expertise valued.
Anthropic
San Francisco, CAStaff-level site reliability engineer responsible for safely deploying and operating safeguards infrastructure across model releases and cloud platforms. The role emphasizes production change management, high-stakes incident response, and automating manual launch and validation processes.
Headway
San Francisco, CAOwn the cloud platform, deployment architecture, container infrastructure, networking, autoscaling, cost controls, and Python runtime health for a high-scale healthcare technology platform. The role requires 8+ years in infrastructure, platform, or SRE work, deep AWS expertise, Terraform experience, and Staff-level cross-team influence.