Build and maintain infrastructure tooling for a large fleet of GPU servers, including provisioning, health monitoring, diagnostics, recovery, storage optimization, and Linux tuning to support AI workloads at scale. Requires 3+ years managing large server fleets, strong Python and deep Linux expertise.
180k – 250k/yr
Hybrid5+ YOEDevOps / SRE
About the role
Key Responsibilities
Build and maintain Python fleet tracking system managing full lifecycle of servers (contracting, procurement, target use, pricing, availability, health, RMAs).
Build server management tooling that automates provisioning, health checks, GPU diagnostics, recovery, and alerting.
Create and maintain metrics, dashboards, and alerting for hardware health across the fleet (GPU errors, disk failures, network issues, thermals).
Leverage AI extensively to build tools and automate alerting and recovery.
Implement and enforce OS-level security: hardening baselines, SELinux/AppArmor policies, SSH key management, vulnerability scanning, and compliance automation.
Manage and optimize distributed and local storage systems (NVMe arrays, NFS, parallel file systems, object storage) supporting model weights, checkpoints, and ephemeral scratch.
Tune Linux systems for AI workloads: kernel parameters, NUMA topology, CPU pinning, hugepages, I/O schedulers, and GPU driver stack optimization (NVIDIA drivers, CUDA, container runtimes).
Develop automated error detection and recovery processes.
Work with partners to solve technical issues.
Requirements
3+ years experience managing bare-metal and cloud-based server fleets at scale (100+ nodes).
Strong software engineering skills in Python for production tooling.
Deep Linux systems knowledge: boot process, kernel tuning, networking, storage, systemd, cgroups, namespaces, performance profiling.
Strong experience with configuration management and infrastructure-as-code (Ansible, Terraform, cloud-init).
Solid understanding of storage technologies: LVM, RAID, NVMe, NFS, Lustre or GPFS, Linux I/O stack tuning.
Familiarity with hardware diagnostics and failure modes (GPUs, NVMe, NICs, memory).
Experience building internal tools or dashboards for infrastructure visibility.
Excellent communication and ability to drive technical decisions across teams.
Self-starter who executes quickly, takes ownership, and constantly seeks improvement.
Nice-to-Haves
Familiarity with network configuration and diagnostics (VLAN, VXLAN, ECMP, BGP, tcpdump).
Experience with NVIDIA GPU infrastructure: driver management, health monitoring, DCGM, NVLink/NVSwitch diagnostics, RDMA, InfiniBand/RoCEv2.
Experience with AMD GPUs.
Experience with bare metal and VM provisioning (PXE/iPXE, Kickstart, libvirt, Qemu/KVM).
Experience with compliance frameworks (SOC 2, ISO 27001).
Platform Engineer building and scaling data ingestion, storage, search, and deployment infrastructure across commercial, FedRAMP High, and IL5 environments. Requires 5+ years in backend/platform/infra roles with deep experience in Postgres, Kubernetes, and TypeScript/Python.
180k – 250k/yr
Hybrid5+ YOEDevOps / SRE
Network Engineer, Design & Engineering
FluidstackNew York, NY +4
Design end-to-end datacenter network architectures for AI training and inference workloads. Own topology selection, fabric design, physical infrastructure integration, and produce deployable HLDs/LLDs across multiple GPU platforms and customer requirements.
180k – 300k/yr
On-site5+ YOEDevOps / SRE
Developer Productivity Engineer
HightouchUnited States
As a Senior Developer Productivity Engineer, you will own the build, test, and deployment processes for a 50+ person engineering team. You will improve monorepo productivity, drive excellence in testing, and support multi-cloud/multi-region infrastructure to enable fast and safe shipping.
180k – 320k/yr
Remote5+ YOEDevOps / SRE
Data Center Network Engineer
BasetenSan Francisco, CA +1
Design and own high-performance data center network infrastructure for GPU clusters, including fabric architecture, cabling, and performance validation. Requires deep experience with InfiniBand, RDMA, or high-performance Ethernet at a senior level.
180k – 360k/yr
Hybrid5+ YOEDevOps / SRE
Infrastructure Engineer (Observability)
Lightning AINew York, NY +2
Builds and operates scalable observability platforms for metrics, logs, traces across GPU, HPC infrastructure. Designs telemetry pipelines, alerting, and multi-tenant systems using Prometheus, Grafana, Kafka; requires 5+ years SRE/infra experience.