# Staff Software Engineer, Inference / Compute Infrastructure Engineering

**Company:** [Together AI](https://hotfix.jobs/companies/together-ai)
**Location:** London, United Kingdom, Amsterdam, Netherlands
**Role:** DevOps / SRE
**Experience:** 7+ years
**Skills:** Go, Python, Rust, Temporal, Cadence, Kubernetes, Event-Driven Systems, Kafka, Nats, Amazon Sqs, Pxe, Ipxe, Redfish, CUDA
**Posted:** 2026-08-20

> Build and operate declarative control planes, durable workflows, and self-healing systems that provision and manage GPU inference infrastructure. The role requires strong software engineering, reconciliation or orchestration experience, and event-driven systems expertise.

## Job Description

## Responsibilities
- Build a provisioning state machine for the full physical-host lifecycle, from discovery and inference bring-up through GPU driver/CUDA installation, health validation, decommissioning, and RMA.
- Design declarative self-service APIs and a control plane for requesting, scaling, and tearing down inference clusters.
- Automate self-healing by detecting degraded or failed nodes, safely draining them, triggering repair or replacement, and restoring healthy capacity.
- Ensure pipeline reliability through idempotency, retries, rollback, and drift detection.
- Partner with inference and ML platform teams to model cluster topology, interconnects, and scheduling constraints.
- Apply production software practices, including strong typing, automated tests, code review, versioning, and CI/CD.

## Requirements
- Strong software engineering experience with Go, Python, Rust, or similar languages.
- Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent.
- Experience building control planes or orchestration systems that model state and reconcile it over time, such as Kubernetes controllers, operators, or custom reconciliation loops.
- Experience designing event-driven systems around message queues, event streams, or pub/sub.
- Experience building internal platforms or APIs for engineering teams, with attention to developer experience.

## Nice to have
- Experience with bare-metal provisioning, including PXE/iPXE, Redfish/IPMI, or BMC.
- Networking fundamentals, including VLANs, BGP, or fabric design.
- GPU or accelerator infrastructure experience.
- Experience with NCCL, CUDA, InfiniBand, or RoCE.
- Experience at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization.
- Systems programming in Rust or Go.

## Similar jobs

- [Staff Software Engineer, Inference / Compute Infrastructure Engineering](https://hotfix.jobs/jobs/25af7190-dc4d-4d85-b2b7-5b41d1dc4405) - Together AI - London, United Kingdom
- [Staff Software Engineer, Observability & Profiling](https://hotfix.jobs/jobs/be415182-687b-4ae9-a1c2-4b70b877514b) - Anthropic - London, United Kingdom - £325k – £390k/yr
- [Staff Software Engineer: Platform SRE](https://hotfix.jobs/jobs/d1231c10-b3f1-48c5-a137-495d17d4f69a) - Flexport - Amsterdam, Netherlands
- [Staff Software Engineer, Inference / Compute Infrastructure Engineering](https://hotfix.jobs/jobs/b1b1bdb1-d3f0-47cd-b94b-7781be7399fc) - Together AI - Remote
- [Staff Software Engineer, AI Reliability Engineering](https://hotfix.jobs/jobs/66b246fd-6b0e-4283-b1ff-c460780b61a2) - Anthropic - London, United Kingdom - £325k – £390k/yr

**Apply:** https://hotfix.jobs/jobs/cf3c46c0-63dd-4681-b117-16200b400d89
**Canonical:** https://hotfix.jobs/jobs/cf3c46c0-63dd-4681-b117-16200b400d89