Skip to content
The Voleon GroupThe Voleon GroupBerkeley, CA

Senior Cluster Site Reliability Engineer

Senior SRE scales and maintains high-performance research compute clusters for machine learning in finance, ensuring uptime, reliability, and observability across on-prem and cloud infrastructure. Requires 5+ years SRE/DevOps experience with HPC frameworks, IaC, and cloud services.

205k – 235k/yr
Remote5+ YOEDevOps / SRE

About the role

Responsibilities

  • Be a first responder in the event of cluster outages or issues. Triage and resolve urgent issues as they arise
  • Ensure a high degree of cluster uptime (measured in multiple nines), and define + track SLAs to quantify reliability
  • Diagnose systemic/recurring patterns of problems, and engineer precision solutions to them in collaboration with engineering teams
  • Develop robust metrics and observability for cluster health and use those metrics to inform your work. Build out custom observability mechanisms when off-the-shelf ones won't do
  • Help software and research teams design policies around fair cluster usage, and help develop enforcement mechanisms for said policies
  • Assist in forecasting cluster growth, and help select appropriate scale-up strategies. Help optimize operations across dimensions of cost and usability

Requirements

  • 5+ years of experience in SRE or DevOps roles, preferably working as a senior engineer or tech lead
  • Knowledge of HPC/batch compute frameworks (Slurm, Kueue, AWS/GCP Batch) and/or machine learning training systems (Kubeflow, MLflow, Horovod)
  • Ability to develop scripts and utilities of moderate complexity in a common scripting language (Python, Ruby, etc.)
  • Familiarity with infrastructure-as-code and configuration management tools (Terraform, Ansible)
  • Experience with cloud infrastructure (AWS or GCP)
  • Familiarity designing and implementing modern observability stacks (Prometheus, Grafana, Loki, ELK, OpenTelemetry)
  • Experience with distributed storage technologies (Lustre, Ceph, S3)
  • Embodies a "system engineer" rather than "system administrator" mindset, thinking systematically and leveraging automation
  • Bachelor degree in computer science

Preferred Qualifications

  • Hands-on experience with HPC frameworks (Slurm, Grid Engine) and Kubernetes-based job orchestrators (Airflow, Kueue, Kubeflow Pipelines), along with other distributed computing frameworks (Ray, Modin, Dask, Spark)
  • Familiarity with ML frameworks (PyTorch/TensorFlow, JAX, Horovod, DeepSpeed)
  • Familiarity with hybrid/on-prem environments
  • Experience with containerization (Docker, Podman, Singularity), particularly for HPC/batch compute environments
  • Experience with HPC networking (InfiniBand, RDMA)
  • Solid security/IAM foundations (Identity management systems, AWS/GCP IAM, Zero Trust)

Skills

SlurmKubernetesTerraformAnsibleAWSGCPPrometheusGrafanaPythonKubeflowDockerLustreCephS3OpenTelemetry

Similar roles

DevOps / SRE jobs
Render

Software Engineer, Network Infrastructure

RenderUnited States

Design, build, and operate Render's core networking stack across data centers and clouds, focusing on Kubernetes and Linux internals, traffic routing, and hybrid connectivity at scale.

204k – 250k/yr
Remote6+ YOEDevOps / SRE
Okta

Senior Manager, Site Reliability Engineering (Federal)

OktaWashington, DC

Lead and mentor multiple SRE teams overseeing Edge networking, Kubernetes platform, CI/CD, observability, and automation tooling for Okta’s high-scale SaaS infrastructure on AWS.

207k – 285k/yr
HybridDevOps / SRE
Vanta

Senior Software Engineer, Enterprise Resilience

VantaUnited States

Build and operate resilient systems for Vanta's FedRAMP and enterprise environments, define reliability frameworks, and partner with teams to ensure scalable, compliant infrastructure using AWS and modern tooling.

207k – 244k/yr
RemoteDevOps / SRE
Render

Product Lead, Infrastructure

RenderUnited States

Lead product vision and roadmap for Render's infrastructure platform supporting millions of developers. Requires 8+ years in product management focused on developer tools, infrastructure, or data products, with strong AI and developer experience interest.

207k – 285k/yr
Remote8+ YOEDevOps / SRE
Zoox

Senior Software Engineer, Build Infrastructure

ZooxFoster City, CA +1

Senior Software Engineer owning and evolving Zoox's Bazel-based build infrastructure for a large multi-language monorepo. Optimize build speeds, develop custom rules/tooling, integrate quality tools, and accelerate developer velocity for safety-critical autonomous vehicle software. Requires 8+ years experience including 4+ years with Bazel.

208k – 263k/yr
Hybrid8+ YOEDevOps / SRE