# Member of Technical Staff - Research Infrastructure Engineer

**Company:** [Black Forest Labs](https://hotfix.jobs/companies/black-forest-labs)
**Location:** Freiburg, Germany
**Role:** DevOps / SRE
**Salary:** €100k – €230k/yr
**Skills:** Python, Bash, Go, Kubernetes, Nvidia Gpu Drivers, Nvidia Gpu Operators, OpenTelemetry, Prometheus, AWS, GCP, Infrastructure As Code, Slurm, Distributed Systems, Gpu Clusters
**Posted:** 2025-09-18

> Build and operate large-scale research infrastructure for generative AI training, including GPU clusters, distributed systems, telemetry, and reliability tooling. The role requires deep cloud infrastructure expertise, Kubernetes, infrastructure as code, and experience operating large-scale training platforms.

## Job Description

## Responsibilities
- Maintain research infrastructure and optimize application- and infrastructure-level components for peak performance.
- Scale infrastructure for growing research demands while maintaining reliability and performance.
- Collaborate with research teams to understand infrastructure needs and design cost-efficient, high-performance solutions.
- Analyze distributed systems at scale to identify and resolve performance bottlenecks and capacity hotspots.
- Build and evolve telemetry and monitoring systems covering infrastructure performance, utilization, and costs across cloud and datacenter fleets.
- Participate in on-call rotations and incident response.

## Requirements
- Experience building or operating large-scale training platforms.
- Experience with large-scale GPU compute clusters.
- Ability to debug performance and reliability issues across large distributed fleets.
- Strong problem-solving skills and ability to work independently.
- Strong communication and collaboration skills.
- Deep knowledge of modern cloud infrastructure, including Kubernetes, infrastructure as code, AWS, and GCP.
- Experience with SLURM.

## Technical Focus
- Python
- Bash
- Go
- Kubernetes
- NVIDIA GPU drivers and operators
- OpenTelemetry
- Prometheus

## Compensation
- EU base salary: €100,000–€230,000 plus equity.
- US base salary: $150,000–$300,000 plus equity.

## Similar jobs

- [Member of Technical Staff](https://hotfix.jobs/jobs/d6c912e7-2f16-4a86-8738-980dd0b47cd0) - Perplexity - Remote - $220k – $405k/yr
- [Software Engineer, Infrastructure](https://hotfix.jobs/jobs/585da47a-e02d-4c63-9816-248a2faa9b5b) - Granica - Remote
- [Platform Engineer - Compute Capacity](https://hotfix.jobs/jobs/4d270716-1d19-4d06-a9a3-4fdc533a14c4) - Supabase - Remote
- [Production Support Engineer](https://hotfix.jobs/jobs/328c6ec8-1e18-42d6-99c0-515d7801c9ac) - Alpaca - Remote
- [ClickHouse Operations Engineer](https://hotfix.jobs/jobs/b2da36d0-2f09-45b4-88c7-996eb12809a8) - PostHog - Remote

**Apply:** https://hotfix.jobs/jobs/c69fd10f-7376-450a-838a-689c1a68d749
**Canonical:** https://hotfix.jobs/jobs/c69fd10f-7376-450a-838a-689c1a68d749