# Software Engineer, Supercomputing

**Company:** [Thinking Machines Lab](https://hotfix.jobs/companies/thinking-machines-lab)
**Location:** San Francisco, CA
**Role:** DevOps / SRE
**Salary:** $350k – $475k/yr
**Skills:** Kubernetes, Slurm, Python, Rust, Linux, CUDA, Nccl, PyTorch, TensorFlow, JAX
**Posted:** 2026-04-02

> Designs, builds, and operates GPU supercomputing environments for large-scale AI training and inference. Automates cluster management, extends orchestration systems, and optimizes performance metrics in collaboration with researchers.

## Job Description

## What You’ll Do
- Operate and automate large GPU clusters including provisioning, imaging, and capacity planning.
- Write software that abstracts cluster management and presents a unified interface for training and inference.
- Extend scheduling/orchestration (Kubernetes, Slurm, or similar) for topology‑aware placement, preemption, quotas, and fair‑share multi‑tenancy.
- Monitor and improve operational metrics of speed, reliability, and error recovery.
- Build reliable storage and artifact paths for datasets, checkpoints, and logs with clear retention and lineage.
- Partner with researchers to unblock scale runs and advise on parallelism and performance trade‑offs.

## Skills and Qualifications
### Minimum qualifications:
- **Bachelor’s degree** or equivalent experience in computer science, engineering, or similar.
- Proficiency in at least one backend language (**Python** or **Rust**).
- Experience operating large‑scale clusters and container orchestration systems (e.g. **Kubernetes** or **Slurm**).
- Comfort operating across the stack and owning projects end-to-end.
- Thrive in a highly collaborative environment involving many, different cross-functional partners and subject matter experts.
- A bias for action with a mindset to take initiative to work across different stacks and different teams where you spot the opportunity to make sure something ships.

### Preferred qualifications:
- Strong systems background: **Linux**, **networking**, and **infrastructure‑as-code**.
- Familiarity with **CUDA**/**NCCL** and performance profiling for distributed training/inference.
- Prior work supporting large‑scale model training or inference environments.
- Understanding of deep learning frameworks (e.g., **PyTorch**, **TensorFlow**, **JAX**) and their underlying system architectures.
- Track record of working in fast-paced environments balancing care with urgency.

## Compensation
Depending on background, skills and experience, the expected annual salary range for this position is **$350,000 - $475,000 USD**.

## Similar roles

- [Reliability Engineer, Supercomputing](https://hotfix.jobs/jobs/e9263e4b-66f2-4203-b8da-b88ba1855dfe) - Thinking Machines Lab - San Francisco, CA - $350k – $475k/yr
- [Network Engineer, Supercomputing](https://hotfix.jobs/jobs/2a952a8c-f34c-4fdd-a219-c8ac22399145) - Thinking Machines Lab - San Francisco, CA - $350k – $475k/yr
- [Performance Engineer, Inference Systems](https://hotfix.jobs/jobs/c4e8b3e8-25ad-4a83-b11d-bebf411d63f1) - Anthropic - San Francisco, CA - $350k – $850k/yr
- [Site Reliability Engineer (SRE)](https://hotfix.jobs/jobs/893a897e-0421-40b8-a8ac-f61c0b30d5ff) - Thinking Machines Lab - San Francisco, CA - $350k – $475k/yr
- [Research Engineer, Infrastructure, Training Systems](https://hotfix.jobs/jobs/9cc8be27-572e-4c19-97d0-8a7ae54e41a8) - Thinking Machines Lab - San Francisco, CA - $350k – $475k/yr

**Apply:** https://hotfix.jobs/jobs/c96dc656-4c41-4964-a025-ca36fd2993aa
**Canonical:** https://hotfix.jobs/jobs/c96dc656-4c41-4964-a025-ca36fd2993aa