Production Engineer, IaaS

Responsibilities

• Own the observability platform: build and operate data pipelines, decoration and correlation engine, and healthcheck framework that make the fleet legible from site down to device and link
• Define and build the API surface for infrastructure: design contracts between production infrastructure and every tool that touches it
• Build the production control plane: unified machine management, actual state inspection, distributed command execution, and the Kubernetes-based infrastructure that underpins it all
• Own fleet state as source of truth: SLOs, site lifecycle state, and integration with internal infrastructure management and customer-facing operations platforms
• Land new hardware into the platform cleanly: ZTP, DHCP, DNS, artifacts — every new XPU generation and site integration goes through IaaS before production

Requirements

• Treat toil as a bug: if something requires a human to do it twice, build the thing that makes it not require a human
• Design APIs that age well; avoid leaky abstractions at scale
• Move toward ambiguity, build the map, and explain it to everyone else
• Learn at a steep slope and reach real competence in unfamiliar domains fast
• Carry a pager without flinching; run incidents, write postmortems, fix systemic causes
• Fluent with AI tooling: LLM APIs, MCP servers, agentic frameworks; drive Claude Code, Cursor, or similar daily
• Shipped production services that other teams depend on at scale; comfortable in any language using AI coding tools

Nice-to-Haves

• Distributed systems and data pipeline engineering
• Time-series observability stacks (Prometheus, Thanos, VictoriaMetrics)
• API design and versioning at scale
• Workflow and orchestration engines (Temporal, Cadence)
• BMC/Redfish or hardware telemetry
• Go, Python, and Postgres