Research Engineer, Synthetic Data
Build synthetic data pipelines and tasks to train frontier AI agents. Requires Python, Docker, Linux, and experience creating realistic, scalable synthetic training data for models and evals.
About the job
Responsibilities
- Work with subject-matter experts to create synthetic tasks for training AI agents across a range of professional and technical domains
- Design synthetic task generation methods that produce diverse, realistic, and learnable tasks
- Build systems and tooling to mutate, validate, and improve synthetic tasks
- Analyze model and agent performance on synthetic tasks to understand what the tasks are teaching and where they fail
- Develop metrics to quantify and understand synthetic task diversity, realism, learnability, etc.
Requirements
- Proficiency in Python, Docker, and Linux environments
- Experience with synthetic data research methods
- Strong understanding of what “good synthetic data” means and its limitations
- Built synthetic data pipelines end-to-end without a fully prescribed roadmap
- Experience working on environments, evals, and benchmarks
Nice-to-Haves
- Detail-oriented and able to spot subtle inconsistencies or edge cases in synthetic data
- Able to reason from first principles about task design, scoring, and failure modes
- Thrive in unstructured problem spaces
- Early-stage startup experience with ability to work independently in fast-paced environments
- Strong communication skills for remote collaboration across time zones
Skills
Python, Docker, Linux, Synthetic Data, AI Agents, Evals, Benchmarks
Similar jobs
ML Engineering jobs
Thinking Machines Lab
San Francisco, CA
Build and operate the engineering systems that support post-training research, including reinforcement learning infrastructure, sandboxed execution, data pipelines, and agent scaffolding. The role requires strong Python and systems engineering skills, project ownership, and a relevant bachelor’s degree or equivalent experience.
OpenAI
San Francisco, CA
Build research infrastructure and tooling that enables AI models to design silicon, including reinforcement learning environments, EDA integrations, evaluations, and experiment workflows. The role requires strong software engineering fundamentals and comfort working across research, tooling, and chip-design systems.
Rollstack
United StatesBuild production AI capabilities for automated slide and document generation, working across LLM applications, data analysis, and content generation. The role requires 3+ years in machine learning and NLP, advanced Python, and experience with LLM frameworks and production systems.
ClickUp
United States
Build and operate large-scale ranking and retrieval systems that power search relevance, including hybrid lexical/vector search, embeddings, query understanding, and permission-aware retrieval. Requires a bachelor's degree and 5+ years of ML engineering experience in ranking or information retrieval.
PathAI
Boston, MADevelop and deploy machine learning models for biomedical research and AI products, collaborating with scientific, engineering, and product teams. Requires an advanced quantitative degree, substantial ML experience, Python proficiency, and experience bringing models into production or research applications.