Technical Lead Manager - Training Runtime, Data Movement

Responsibilities

• Design and build a unified dataset read platform for multiple current and future training frameworks.
• Define dataset APIs, storage-format expectations, registration/versioning, and migration paths that make data access reproducible and maintainable.
• Build reliability into the read path, including stateful iteration, caching, fast restart, recovery, and clear operational contracts.
• Build terminal and web-based visualizers that let teams inspect text, multimodal, and reinforcement learning data late in the pipeline.
• Write and review production code in core data loading, service, caching, and reliability paths.
• Partner with teams working on training frameworks, reinforcement learning, multimodal models, storage, runtime, and cluster infrastructure.

Requirements

• Have built or owned dataset, data loading, storage, or distributed training infrastructure at large scale (e.g. torch.utils.data).
• Care equally about API design, debugging ergonomics, performance, and bit-level correctness.
• Understand the failure modes of large distributed training jobs and know how data systems can create or prevent them.
• Have experience with stateful iterators, checkpoint/restart semantics, caching, remote services, or high-throughput storage reads.
• Are comfortable working across Python and lower-level systems code; Rust or C++ experience is useful but not required.
• Have worked with multimodal, video, reinforcement learning, or pretraining data pipelines where small data bugs are expensive and hard to diagnose.
• Can lead through code and technical judgment before a team exists, and can later manage engineers without losing the hands-on edge.
• Obsess over developer experience by eliminating friction, such as manual preprocessing scripts and niche cluster-specific bugs.