Member of Technical Staff - Pretraining
Leads frontier-scale pretraining research for multimodal image, video, and audio foundation models, shaping architectures, objectives, data strategies, and distributed systems. The role requires prior ownership of production-grade foundation-model pretraining, deep Python and PyTorch expertise, and strong experience with visual generative models.
About the job
Responsibilities
- Lead large-scale pretraining experiments for multimodal image, video, and audio foundation models, including architecture, objective functions, and scaling strategies.
- Develop and evaluate novel ideas across architectures, optimizers, and training algorithms.
- Contribute across the full stack, from low-level GPU and systems optimizations to research code and high-level model design.
- Lead focused research projects independently and drive larger cross-team initiatives.
Requirements
- Led or co-owned pretraining for a foundation model—image, video, language, or multimodal—that shipped to production or a major release.
- Experience making architectural decisions involving attention patterns, modulation schemes, loss formulations, and tokenization strategies.
- Deep experience with large-scale distributed training, including FSDP, tensor parallelism, pipeline parallelism, and multi-node runs at 500+ GPUs.
- Experience debugging loss spikes, NaNs, throughput regressions, and silent correctness issues at scale.
- Strong intuition for architecture and objective design.
- Track record of shipping, demonstrated through top-venue publications paired with production impact or unambiguous production wins at a frontier lab.
- Deep Python and PyTorch proficiency, including the ability to read and modify low-level training code.
- Familiarity with visual generative models.
Compensation
- Base annual salary: €130,000–€340,000, plus equity.
Skills
Python, PyTorch, Fsdp, Tensor Parallelism, Pipeline Parallelism, Distributed Training, Gpu Optimization, Multimodal Models, Visual Generative Models, Foundation Models, Pretraining, Tokenization, Optimizers, Attention Mechanisms, Training Algorithms
Similar jobs
AI Research jobs
Black Forest Labs
Freiburg, Germany
Advances vision-language models and integrates multimodal capabilities with FLUX diffusion and flow pipelines. The role requires demonstrated VLM pretraining or substantial architectural advancement, strong research or production results, and multi-node distributed training experience.
Vanta
Remote
Build Vanta’s organizational intelligence layer by shipping prototypes, internal tools, and AI agent workflows that make cross-source data useful to EPD, GTM, and other teams. The role requires recent hands-on LLM product work, independent problem scoping, and strong judgment around AI quality, reliability, cost, and latency.
AI Digest
Remote
Conduct research on long-horizon, multi-agent AI behavior by designing agent environments, analyzing large-scale data, and running experiments. The role requires strong research judgment, rapid execution, independence, and familiarity with current AI developments.
AI Digest
Remote
Build, optimize, and evaluate long-running and multi-agent AI systems, along with tools for monitoring and analyzing their real-world behavior. The role requires software engineering experience with coding agents, strong independence, and familiarity with current AI developments.
Improbable
Remote
Conduct applied research on AI agents, designing experiments and evaluation systems to improve reliability, context retention, and multi-step task completion. The role requires strong AI/ML research, engineering, experimental design, and communication skills.