# Systems Architect AI/ML Infrastructure

**Company:** [Deepgram](https://hotfix.jobs/companies/deepgram)
**Location:** Remote
**Role:** DevOps / SRE
**Salary:** $160k – $220k/yr
**Experience:** 7+ years
**Skills:** Kubernetes, AWS, GPU, Storage Systems, Multi-Cloud, Finops, Slurm, Ray, Bare Metal, Networking
**Posted:** 2026-04-06

> Designs end-to-end infrastructure architecture for AI/ML workloads, including multi-cloud strategies, GPU orchestration, storage for massive datasets, and capacity planning to support production inference and research training at scale. Requires 7+ years in systems architecture with deep expertise in Kubernetes, AWS, and GPU infrastructure.

## Job Description

## What You'll Do

- Define and drive the end-to-end infrastructure architecture for Deepgram's AI/ML workloads across production inference and research training
- Design multi-cloud and hybrid infrastructure strategies that balance performance, reliability, cost, and vendor flexibility
- Architect compute orchestration systems that efficiently schedule and manage GPU and CPU workloads across heterogeneous infrastructure
- Design storage architectures that handle the massive datasets required for speech and audio ML -- from high-throughput training data pipelines to low-latency model serving
- Lead capacity planning across all infrastructure dimensions, modeling growth and ensuring Deepgram can scale ahead of demand
- Drive cost optimization and FinOps practices, identifying opportunities to reduce infrastructure spend without compromising performance or reliability
- Design burstable, elastic training infrastructure that can scale up for large training runs and scale down to minimize idle cost
- Architect research compute infrastructure that gives ML teams the resources they need while maintaining operational efficiency
- Establish architectural standards, design review processes, and technical documentation practices for infrastructure decisions
- Collaborate with engineering leadership to align infrastructure strategy with product roadmap and business objectives
- Evaluate emerging hardware, cloud services, and infrastructure technologies for potential adoption

## Requirements

- 7+ years of experience in infrastructure engineering, systems architecture, or a senior technical role focused on large-scale infrastructure
- Proven experience designing multi-cloud architectures spanning **AWS** and at least one other major cloud provider or on-premises environment
- Deep expertise in storage system design -- block, object, and file storage, including performance tuning for large-scale data workloads
- Strong experience with compute orchestration using **Kubernetes**, and an understanding of how to schedule diverse workloads efficiently
- Hands-on experience with **GPU** infrastructure -- procurement considerations, cluster design, driver and runtime management
- Track record of capacity planning and infrastructure scaling for high-growth environments
- Ability to communicate complex architectural decisions clearly to both technical and non-technical stakeholders
- Strong understanding of networking fundamentals as they relate to infrastructure architecture

## Nice-to-Haves

- Direct experience architecting infrastructure for ML training workloads -- distributed training, large dataset management, experiment infrastructure
- Background in cost optimization and **FinOps** practices for large-scale cloud and bare metal infrastructure
- Experience operating and managing bare metal infrastructure in colocation facilities
- Expertise in network architecture design, including high-bandwidth GPU interconnects and global traffic routing
- Experience with infrastructure modeling and simulation for capacity planning
- Familiarity with **Slurm**, **Ray**, or other HPC/ML job scheduling systems
- Understanding of power, cooling, and physical infrastructure considerations for GPU-dense deployments

## Similar roles

- [Senior Site Reliability Engineer](https://hotfix.jobs/jobs/076df34d-0d8e-4f15-8a79-eb559a8e5f52) - Talkiatry - Remote - $160k – $185k/yr
- [Senior Infrastructure Engineer](https://hotfix.jobs/jobs/7fad3112-9910-402c-b20e-17b838f39c55) - Aurelian - Seattle, WA - $160k – $220k/yr
- [Senior Microsoft Cloud Infrastructure Engineer](https://hotfix.jobs/jobs/2c893053-22db-4cbf-a24f-ab191f6ca0d6) - Crusoe - San Francisco, CA - $160k – $195k/yr
- [Lead DevOps Engineer](https://hotfix.jobs/jobs/c42aca88-adaa-4b4a-9d56-58f3b9521efe) - Octus - New York, NY - $160k – $215k/yr
- [Senior Central Cloud Infrastructure Engineer](https://hotfix.jobs/jobs/87716575-2734-4ad0-ae9c-ddedbc91b04b) - Metropolis - New York, NY - $160k – $200k/yr

**Apply:** https://hotfix.jobs/jobs/860f2c79-63d7-4478-a402-956f668b6e1a
**Canonical:** https://hotfix.jobs/jobs/860f2c79-63d7-4478-a402-956f668b6e1a