# Staff Production Engineer (Operational Excellence)

**Company:** [Crusoe](https://hotfix.jobs/companies/crusoe)
**Location:** San Francisco, CA, Sunnyvale, CA
**Role:** DevOps / SRE
**Salary:** $209k – $253k/yr
**Experience:** 8+ years
**Skills:** Kubernetes, Prometheus, Grafana, Terraform, Ansible, Linux, Python, Go, AWS, GCP
**Posted:** 2025-12-05

> Leads reliability and operational excellence for Crusoe's GPU cloud platform, driving incident response, observability with Prometheus/Grafana, automation to reduce toil, and SLOs for AI/HPC workloads. Requires 8+ years in SRE/production engineering, Kubernetes, Linux, and programming in Python/Go.

## Job Description

## What You'll Be Working On
- Lead cross-functional efforts to define and evolve availability metrics for Crusoe's cloud platform, including establishing, measuring, and improving SLIs and SLOs
- Drive production incident response, diagnosing and resolving service disruptions while leading post-incident reviews and root cause analysis
- Architect, operate, and improve observability across Crusoe's infrastructure using tools such as Prometheus, Grafana, Alertmanager, and OpenTelemetry
- Identify reliability risks, performance bottlenecks, and early indicators of potential production issues across distributed systems
- Design and develop automation and tooling that reduces operational toil, improves recovery times, and enables self-healing infrastructure
- Partner with compute, networking, storage, and platform teams to strengthen service resilience and disaster recovery capabilities
- Define and champion operational processes, knowledge sharing, and reliability best practices across the engineering organization
- Mentor and grow junior and mid-level engineers, helping build technical depth across the team

## What You'll Bring to the Team
- Bachelor's degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience)
- 8+ years of experience in Production Engineering, SRE, or large-scale infrastructure operations
- Demonstrated experience supporting GPU workloads, HPC environments, or latency/throughput-sensitive distributed systems
- Deep knowledge of Linux/Unix systems, including debugging complex issues across kernel and user space
- Strong understanding of modern cloud infrastructure fundamentals including **Kubernetes**, distributed systems, virtualization, and cloud platforms (**AWS**/**GCP**)
- Proven track record with incident management practices and reliability frameworks (SRE, ITIL, or similar)
- Hands-on experience with monitoring and observability tools such as **Prometheus** and **Grafana**
- Experience with infrastructure-as-code and configuration management tools such as **Terraform** or **Ansible**
- Proficiency in scripting or programming with languages such as **Go**, **Python**, **C**, or **C++**
- Exceptional communication skills and the ability to influence and collaborate across engineering teams
- Ability to remain calm and effective while troubleshooting complex issues in high-impact production environments
- A growth mindset and strong commitment to reliability engineering, automation, and operational excellence

**Bonus Points:**
- Experience leading **Kubernetes** or container orchestration platforms at scale
- Exposure to change management processes, operational readiness reviews, or structured root cause analysis
- Experience designing self-healing systems, automated remediation, or event-driven operational tooling
- Interest in scaling AI or HPC infrastructure and solving reliability challenges in GPU-heavy environments
- Passion for mentorship, growing teams, and developing deep expertise in Production Engineering

## Compensation
- Compensation will be paid in the range of $209,000 – $253,000 + Bonus
- Restricted Stock Units are included in all offers

## Similar roles

- [Staff Production Engineer, Compute](https://hotfix.jobs/jobs/7a9a870e-9394-42b9-b5f9-423d777df634) - Crusoe - San Francisco, CA - $209k – $253k/yr
- [Staff Software Engineer, Developer Experience](https://hotfix.jobs/jobs/e13687f0-a569-475f-be3a-f5b0178c8759) - Crusoe - San Francisco, CA - $209k – $253k/yr
- [Staff Infrastructure Engineer](https://hotfix.jobs/jobs/bd50f619-96c5-4ab7-b28e-a3646ed6c4a6) - Crusoe - San Francisco, CA - $208k – $253k/yr
- [Staff Platform Engineer, AI Enablement](https://hotfix.jobs/jobs/9ff4a9f6-a770-4a9d-98d9-b69541885a2e) - Zocdoc - Remote - $210k – $270k/yr
- [Staff Software Engineer, App Platform](https://hotfix.jobs/jobs/e2ea84b0-4fed-4d58-8632-902515a3dde4) - Fieldguide - Remote - $210k – $265k/yr

**Apply:** https://hotfix.jobs/jobs/f20d06a8-22d7-4624-a609-698fd8923f54
**Canonical:** https://hotfix.jobs/jobs/f20d06a8-22d7-4624-a609-698fd8923f54