Software Engineer, Pretraining

Responsibilities

Data Quality

• Build and own high-throughput, fully telemetered data pipelines with end-to-end traceability.
• Train and ship high-throughput models for classifying, ranking, filtering, cleaning, and identifying data.
• Design and run scaling-ladder experiments on data mixtures, repeatability, and quality depth.
• Partner with data acquisition and training teams to identify missing or low-quality sources and measure their impact.
• Write performance-critical code and design rigorous experiments.

Data Platform

• Build platforms that transform raw web, code, multimodal, and acquired data into training-ready datasets.
• Own pipelines, orchestration, and tooling for fast, reliable, observable, and reproducible pretraining data iteration.
• Create signals for data quality, lineage, freshness, and pipeline health.
• Partner with initial training, crawling, data quality, and acquisition teams to turn data ideas into measurable improvements.

Crawling

• Build and scale web-crawling systems that discover, schedule, fetch, and parse documents across the open web.
• Improve URL seeding, scoring, and fair host scheduling.
• Increase crawl success and parsing quality by addressing antibot failures and improving extractors.
• Debug and harden crawl infrastructure for availability, recovery, and ingestion lag.
• Automate delivery of crawl datasets into downstream data pipelines.

Requirements

• Strong infrastructure or data-platform background, ideally with significant depth in a specialized area.
• Demonstrated ability to architect and ship end-to-end systems with high ownership.
• Ability to debug complex systems independently and work alongside AI agents.
• Strong intuition for large-scale distributed systems.
• Interest in how pretraining data shapes model quality.