Senior/Staff Kubernetes Infrastructure Engineer

Responsibilities

• Design, automate, validate, and deliver the complete lifecycle of customer compute environments, from provisioning through upgrades, recovery, and decommissioning.
• Use AI to automate and accelerate infrastructure delivery and operations.
• Provision dedicated Kubernetes and Slurm clusters tailored to customer workloads.
• Build and maintain Linux images and automated OS-provisioning workflows.
• Operate the NVIDIA GPU stack, including drivers, GPU Operator, NVIDIA Container Toolkit, device plugins, MIG, and GPU monitoring.
• Design Kubernetes and data-center networking using Cilium/Calico, MetalLB, VLAN, VXLAN, BGP, and ECMP.
• Configure distributed and shared storage for high-performance workloads.
• Build monitoring, alerting, diagnostics, and automated recovery for customer environments.
• Develop reusable tooling, standards, documentation, and runbooks.
• Collaborate with customers and internal teams to translate workload requirements into infrastructure designs.

Requirements

• 5+ years of experience building and operating production Linux infrastructure.
• Strong production experience with Kubernetes on bare metal, including bootstrapping, upgrades, highly available control planes, etcd, containerd, CNI, CSI, ingress, load balancing, observability, security, and troubleshooting.
• Experience with Linux virtualization using KVM/QEMU, libvirt, and VFIO device passthrough.
• Experience operating NVIDIA GPUs on Linux and Kubernetes, including drivers, container runtimes, device plugins, GPU Operator, and GPU telemetry.
• Strong networking fundamentals, including TCP/IP, L2/L3, VLANs, routing, and packet-level troubleshooting with tcpdump and Wireshark.
• Practical scripting experience.
• Experience with configuration-management tools such as Ansible.
• Ability to diagnose complex, cross-layer infrastructure issues.
• Strong communication skills and the ability to drive technical decisions across teams.
• Track record of moving quickly, taking ownership, and continuously improving systems.

Nice to Have

• Production Slurm experience.
• High-performance networking: NVLink/NVSwitch, InfiniBand, RoCEv2, GPUDirect RDMA, NCCL, and IMEX.
• Hugepages, NUMA, and CPU pinning.
• SR-IOV and DPDK.
• Distributed storage: Ceph, Lustre, and Weka.
• KubeVirt and OpenStack.
• IPsec, WireGuard, and Tailscale.
• VXLAN, BGP, and ECMP.
• Bare-metal management: BMC, IPMI, Redfish, PXE/iPXE, Kickstart, and cloud-init.
• Network automation with NetBox, Nautobot, and Nornir.
• AI training, inference, or distributed GPU workload infrastructure.
• Python or Go proficiency.