Skip to content
Thinking Machines LabThinking Machines LabSan Francisco, CA

Software Engineer, Supercomputing

Designs, builds, and operates GPU supercomputing environments for large-scale AI training and inference. Automates cluster management, extends orchestration systems, and optimizes performance metrics in collaboration with researchers.

350k – 475k/yr
On-siteDevOps / SRE

About the role

What You’ll Do

  • Operate and automate large GPU clusters including provisioning, imaging, and capacity planning.
  • Write software that abstracts cluster management and presents a unified interface for training and inference.
  • Extend scheduling/orchestration (Kubernetes, Slurm, or similar) for topology‑aware placement, preemption, quotas, and fair‑share multi‑tenancy.
  • Monitor and improve operational metrics of speed, reliability, and error recovery.
  • Build reliable storage and artifact paths for datasets, checkpoints, and logs with clear retention and lineage.
  • Partner with researchers to unblock scale runs and advise on parallelism and performance trade‑offs.

Skills and Qualifications

Minimum qualifications:

  • Bachelor’s degree or equivalent experience in computer science, engineering, or similar.
  • Proficiency in at least one backend language (Python or Rust).
  • Experience operating large‑scale clusters and container orchestration systems (e.g. Kubernetes or Slurm).
  • Comfort operating across the stack and owning projects end-to-end.
  • Thrive in a highly collaborative environment involving many, different cross-functional partners and subject matter experts.
  • A bias for action with a mindset to take initiative to work across different stacks and different teams where you spot the opportunity to make sure something ships.

Preferred qualifications:

  • Strong systems background: Linux, networking, and infrastructure‑as-code.
  • Familiarity with CUDA/NCCL and performance profiling for distributed training/inference.
  • Prior work supporting large‑scale model training or inference environments.
  • Understanding of deep learning frameworks (e.g., PyTorch, TensorFlow, JAX) and their underlying system architectures.
  • Track record of working in fast-paced environments balancing care with urgency.

Compensation

Depending on background, skills and experience, the expected annual salary range for this position is $350,000 - $475,000 USD.

Skills

KubernetesSlurmPythonRustLinuxCUDANcclPyTorchTensorFlowJAX

Similar roles

DevOps / SRE jobs
Thinking Machines Lab

Reliability Engineer, Supercomputing

Thinking Machines LabSan Francisco, CA

Ensure reliability of large GPU supercomputing clusters by diagnosing hardware/firmware/OS issues, automating monitoring, driving firmware rollouts, and working directly with vendors.

350k – 475k/yr
On-siteDevOps / SRE
Thinking Machines Lab

Network Engineer, Supercomputing

Thinking Machines LabSan Francisco, CA

Own and debug multi-thousand-GPU network fabric (RDMA/RoCE, NVLink/NVSwitch) for large-scale AI training and inference. Requires backend language proficiency, large-scale cluster experience, and cross-stack ownership.

350k – 475k/yr
On-siteDevOps / SRE
Anthropic

Performance Engineer, Inference Systems

AnthropicSan Francisco, CA +2

Performance engineer focused on cross-layer investigations of Anthropic's inference fleet for Claude, optimizing throughput, latency, reliability, and correctness while building observability and partnering with kernel and serving teams.

350k – 850k/yr
HybridDevOps / SRE
Thinking Machines Lab

Site Reliability Engineer (SRE)

Thinking Machines LabSan Francisco, CA

Site Reliability Engineer drives end-to-end reliability for AI fine-tuning platform Tinker, including SLOs, monitoring, incident response, and multi-tenant GPU scheduling. Requires distributed systems experience, software proficiency for reliability, and production incident handling.

350k – 475k/yr
On-siteDevOps / SRE
Thinking Machines Lab

Research Engineer, Infrastructure, Training Systems

Thinking Machines LabSan Francisco, CA

Designs and optimizes distributed training systems scaling across thousands of GPUs for large AI models. Requires strong systems engineering, PyTorch/JAX expertise, and collaborative mindset to boost research productivity.

350k – 475k/yr
On-siteDevOps / SRE