# Infrastructure Engineer (GPU & Compute)

**Company:** [Lightning AI](https://hotfix.jobs/companies/lightning-ai)
**Location:** Remote
**Role:** DevOps / SRE
**Salary:** $180k – $200k/yr
**Experience:** 5+ years
**Skills:** Linux, Python, Nvidia Dcgm, Bare-Metal Provisioning, Pxe, Ipmi, Redfish, InfiniBand, Nvlink, Virtualization
**Posted:** 2026-05-01

> Owns GPU diagnostics, validation workflows, and automation for bare-metal infrastructure supporting AI/ML workloads. Requires 5+ years in systems engineering with strong Linux, Python, and NVIDIA tools expertise.

## Job Description

## What You'll Do

### Systems, Image & Validation Infrastructure
- Own and evolve systems for image management, deployment, and validation across bare-metal infrastructure
- Run and maintain test clusters used for system validation, diagnostics, and bring-up
- Validate firmware, drivers, and OS images across compute and GPU-enabled systems
- Support hardware qualification efforts for next-generation platforms

### GPU Diagnostics & Performance
- Own GPU diagnostics and validation workflows across large-scale infrastructure
- Diagnose and resolve complex issues across GPUs, drivers, OS, and hardware layers
- Analyze system and GPU performance using tools such as **NVIDIA DCGM**
- Identify failure patterns and drive improvements in system stability and validation coverage

### Automation & Tooling
- Build and maintain automation for provisioning, validation, and system bring-up
- Develop **Python**-based tools and workflows to improve efficiency and reduce manual operational overhead
- Improve the reliability, repeatability, and scalability of image pipelines and validation systems

### Systems & Operations
- Manage and operate **Linux**-based systems in production and validation environments
- Manage virtualization technology
- Support bare-metal provisioning workflows, including **PXE** and image-based systems
- Interface with hardware management systems (e.g., **IPMI**, **Redfish**) for monitoring and debugging

### Cross-Functional Collaboration
- Partner with Infrastructure, Hardware, and Data Center teams on system bring-up and validation
- Collaborate with platform and ML teams to ensure systems meet workload requirements
- Contribute to best practices for provisioning, diagnostics, and lifecycle management of infrastructure

## What You'll Need

### Required Qualifications
- 5+ years of experience in infrastructure engineering, systems engineering, or related roles
- Strong **Linux** systems experience in production environments
- Hands-on experience with GPU-enabled systems and tools such as **NVIDIA DCGM**
- Familiarity with bare-metal provisioning and system bring-up workflows
- Proficiency in **Python** or similar scripting/programming languages for automation
- Ability to debug complex issues across hardware, OS, GPUs, and system software

### Ideal Experience
- Experience with high-performance interconnects (e.g., **InfiniBand**, **NVLink**)
- Experience with **PXE** boot environments, LiveCD systems, or image-based provisioning workflows
- Experience with hardware management interfaces such as **iDRAC**, **IPMI**, or **Redfish**
- Data center operations experience, including working with physical hardware
- Experience supporting **AI/ML** or **HPC** workloads at scale
- Experience with GPU validation frameworks or large-scale hardware qualification processes

## Similar roles

- [Platform Engineer](https://hotfix.jobs/jobs/14231c8e-abad-4a41-94e3-d41fb2ede6f5) - Govsignals - New York, NY - $180k – $250k/yr
- [Network Engineer, Design & Engineering](https://hotfix.jobs/jobs/48ff3694-5232-4d66-b43c-a091b96c8902) - Fluidstack - New York, NY - $180k – $300k/yr
- [Developer Productivity Engineer](https://hotfix.jobs/jobs/9a60db26-80d5-4b2a-b613-56ae58d49794) - Hightouch - Remote - $180k – $320k/yr
- [Data Center Network Engineer](https://hotfix.jobs/jobs/473ec43a-a5df-4433-99bd-4053f3e6f6bc) - Baseten - San Francisco, CA - $180k – $360k/yr
- [Infrastructure Engineer (Observability)](https://hotfix.jobs/jobs/098c81d9-0827-4baa-a70b-444cc5b04f37) - Lightning AI - Remote - $180k – $200k/yr

**Apply:** https://hotfix.jobs/jobs/2dd5ac53-2601-447b-af6d-e643a08de3ad
**Canonical:** https://hotfix.jobs/jobs/2dd5ac53-2601-447b-af6d-e643a08de3ad