Skip to content
CognitionCognitionSan Francisco, CA

Research Engineer, Infrastructure

Builds and owns distributed training infrastructure, experiment orchestration, data pipelines, and performance optimizations for large-scale AI research on GPU clusters. Requires deep systems expertise, Python/C++/PyTorch proficiency, and ML understanding to accelerate frontier research.

Salary not listed
On-siteDevOps / SRE

About the role

Responsibilities

  • Build and own distributed training infrastructure for large-scale GPU clusters, including job launchers, checkpointing, recovery, fault tolerance, and monitoring.
  • Own infrastructure for scaling agent rollouts in VM sandboxes at RL training scales.
  • Profile and optimize training throughput: data loading, communication, memory, compute efficiency to improve step time and MFU.
  • Design experiment orchestration and tooling to launch, track, and analyze experiments.
  • Build high-throughput, reliable data pipelines for training and evaluation.
  • Debug and resolve training failures across GPUs, networking, numerics, and data.
  • Implement and optimize parallelism strategies: data, tensor, pipeline, sequence.
  • Anticipate research needs and build scaling infrastructure proactively.

Requirements

  • Deep experience building/operating distributed training systems for large models.
  • Strong systems engineering: distributed systems, networking, storage, performance reasoning.
  • Proficiency in Python, C++; systems-level PyTorch or equivalent.
  • Hands-on GPU performance profiling, memory optimization, compute efficiency.
  • Experience with parallelism strategies for large model training.
  • Track record building tooling to accelerate research workflows.
  • Strong debugging in complex distributed systems.
  • ML knowledge to engage with researchers.

Skills

PythonC++PyTorchDistributed SystemsGpu ProgrammingParallelismData PipelinesExperiment OrchestrationPerformance ProfilingNetworking

Similar roles

DevOps / SRE jobs
Forus

Software Engineer, Platform & Infrastructure

ForusNew York, NY

Software Engineer owning Forus' compute platform (EKS/Kubernetes), data layer (Postgres, OpenSearch, BigQuery), cloud cost optimization, reliability (SLOs, observability), and IaC primitives in a regulated healthcare environment. Requires production Kubernetes at scale, deep AWS/Terraform expertise, and database migration experience.

Salary not listed
On-site5+ YOEDevOps / SRE
Forus

Software Engineer, Developer Productivity

ForusNew York, NY

Software Engineer owning CI/CD, build systems, and infrastructure for AI coding agents to accelerate the entire engineering organization's velocity at an AI-powered healthcare network company. Requires strong infrastructure instincts, Python/Bash expertise, and experience driving developer productivity and adoption of agentic workflows.

Salary not listed
On-siteDevOps / SRE
Runloop

Site Reliability Engineer

RunloopSan Francisco, CA

Site Reliability Engineer responsible for the reliability, observability, performance, and security of a core AI agent platform including code sandboxes. Requires 5+ years software engineering experience (3+ in SRE/DevOps), strong CS fundamentals, expertise in containers, cloud IaC, monitoring, and distributed systems.

Salary not listed
On-site5+ YOEDevOps / SRE
Applied Intuition

Build & Release Engineer

Applied IntuitionSunnyvale, CA

Build and maintain CI/CD pipelines, own end-to-end software releases and artifact management with Artifactory, and develop the internal Release Management portal as a hands-on software engineer. Requires 3+ years software development experience and strong CI/CD and Git expertise.

118k – 200k/yr
On-site3+ YOEDevOps / SRE
Lyft

Software Engineer, Async Platform

LyftSeattle, WA

Software engineer building and maintaining a highly scalable asynchronous platform at Lyft. Design, develop and operate tooling for reliability, scalability and efficiency of distributed systems and infrastructure. Requires 5+ years in software development, automation and systems engineering with experience in Go/Python and cloud platforms.

118k – 147k/yr
Hybrid5+ YOEDevOps / SRE