Skip to content
CrusoeCrusoeSan Francisco, CA

Senior Production Engineer, Operational Excellence

Senior Production Engineer ensures reliability, scalability, and performance of GPU cloud infrastructure powering AI workloads. Drives observability, incident response, automation, and operational improvements in large-scale distributed systems.

172k – 209k/yr
On-site5+ YOEDevOps / SRE

About the role

Responsibilities

  • Collaborate with cross-functional teams to define and evolve availability metrics for Crusoe’s cloud platform, including establishing, measuring, and improving SLIs and SLOs
  • Participate in production incident response, diagnosing and resolving service disruptions while contributing to post-incident reviews and root cause analysis
  • Build, operate, and improve observability across Crusoe’s infrastructure using tools such as Prometheus, Grafana, Alertmanager, and OpenTelemetry
  • Identify reliability risks, performance bottlenecks, and early indicators of potential production issues across distributed systems
  • Develop automation and tooling that reduces operational toil, improves recovery times, and enables self-healing infrastructure
  • Partner with compute, networking, storage, and platform teams to strengthen service resilience and disaster recovery capabilities
  • Contribute to improving operational processes, knowledge sharing, and reliability best practices across the engineering organization

Requirements

  • 5+ years of experience in Production Engineering, SRE, or large-scale infrastructure operations
  • Experience supporting GPU workloads, HPC environments, or latency/throughput-sensitive distributed systems
  • Strong knowledge of Linux/Unix systems, including debugging complex issues across kernel and user space
  • Previous experience in Infrastructure roles building or managing compute, storage or networking platforms
  • Understanding of modern cloud infrastructure fundamentals including Kubernetes, distributed systems, virtualization, and cloud platforms (AWS/GCP)
  • Familiarity with incident management practices and reliability frameworks (SRE, ITIL, or similar)
  • Experience with monitoring and observability tools such as Prometheus and Grafana
  • Familiarity with infrastructure-as-code and configuration management tools such as Terraform or Ansible
  • Scripting or programming experience with languages such as Go, Python, C, or C++
  • Strong communication skills and the ability to collaborate across engineering teams
  • Ability to remain calm and effective while troubleshooting complex issues in high-impact production environments

Nice-to-Haves

  • Experience working with Kubernetes or container orchestration platforms at scale
  • Exposure to change management processes, operational readiness reviews, or structured root cause analysis
  • Experience designing self-healing systems, automated remediation, or event-driven operational tooling
  • Interest in scaling AI or HPC infrastructure and solving reliability challenges in GPU-heavy environments

Compensation

  • Base salary range: $172,000 – $209,000 + Bonus
  • Restricted Stock Units included

Skills

PrometheusGrafanaAlertmanagerOpenTelemetryKubernetesLinuxTerraformAnsiblePythonGoAWSGCPSREGpu WorkloadsHpc

Similar roles

DevOps / SRE jobs
Crusoe

Senior Virtualization Validation Engineer

CrusoeSan Francisco, CA +1

Validates large-scale multi-node GPU clusters using QEMU and Cloud Hypervisor, focusing on interconnects like NVLink/InfiniBand, collective communications (NCCL/RCCL), and performance in virtualized AI/HPC environments. Requires 5+ years experience, virtualization expertise, and Linux kernel knowledge.

173k – 210k/yr
On-site5+ YOEDevOps / SRE
Illumio

Sr. Site Reliability Engineer

IllumioSunnyvale, CA

Senior Site Reliability Engineer responsible for ensuring reliability, scalability, and performance of Illumio's AWS and Azure cloud infrastructure. Lead monitoring, incident response, on-call support, automation, and continuous improvement initiatives in a cybersecurity SaaS environment.

170k – 196k/yr
On-site5+ YOEDevOps / SRE
Render

Software Engineer, Compute Infrastructure

RenderSan Francisco, CA

Build and own core compute infrastructure for Render's cloud platform, including Kubernetes clusters on hyperscalers and bare metal. Design, scale, debug, and optimize large-scale orchestration, scheduling, and distributed systems with deep Kubernetes and systems expertise.

170k – 290k/yr
Remote7+ YOEDevOps / SRE
Skydio

Senior Software Engineer, Developer Productivity Cloud Infrastructure

SkydioSan Mateo, CA

Senior engineer focused on developer productivity and cloud infrastructure. Designs scalable internal tools, re-architects build systems, and improves CI/CD workflows using Terraform, Go/Python/C++.

170k – 240k/yr
Hybrid5+ YOEDevOps / SRE
Sigma

Senior Software Engineer - Observability and Reliability

SigmaSan Francisco, CA +1

Build observability tools and platforms (metrics, logging, tracing, alerting) using Go, OpenTelemetry, and Kubernetes. Requires 5+ years experience building high-quality software that other engineers use.

170k – 240k/yr
On-site5+ YOEDevOps / SRE