# Research Lead, Training Insights

**Company:** [Anthropic](https://hotfix.jobs/companies/anthropic)
**Location:** San Francisco, CA, New York, NY
**Role:** AI Research
**Salary:** $850k – $850k/yr
**Skills:** LLMs, Reinforcement Learning, Evaluation Methodologies, Long-Horizon Evaluations, Python, Machine Learning, Ai Safety, Benchmarking, Red Teaming, Experimental Design
**Posted:** 2026-03-06

> Leads strategy and execution for measuring AI model capabilities across training and deployment, developing novel long-horizon evaluations and leading a team of researchers. Requires experience in LLM evaluations, technical leadership, and cross-team collaboration in fast-paced AI research environments.

## Job Description

## Responsibilities:
- Build new novel and long-horizon evaluations
- Develop novel measurement approaches for understanding how model capabilities emerge and evolve during RL training
- Lead strategic evaluation coverage across the company
- Shape the evaluation narrative for model releases
- Lead and mentor a small team of researchers and research engineers, setting research direction and fostering a culture of rigorous, creative research
- Design evaluation frameworks that balance scientific rigor with the practical demands of production training schedules
- Build and maintain relationships across Anthropic's research organization to ensure evaluation insights inform training and deployment decisions
- Contribute to the broader research community through publications, open-source contributions, or external engagement on evaluation best practices

## You may be a good fit if you:
- Have significant experience designing and running evaluations for large language models or similar complex ML systems
- Have led technical projects or teams, either formally or through sustained ownership of critical research directions
- Are equally comfortable designing experiments and writing code—you can move between research and implementation fluidly
- Think strategically about what to measure and why, not just how to measure it
- Can synthesize information across multiple teams and workstreams to form a coherent picture of model capabilities
- Communicate complex technical findings clearly to both technical and non-technical audiences
- Are results-oriented and thrive in fast-paced environments where priorities shift based on research findings
- Care deeply about AI safety and want your work to directly influence how capable AI systems are developed and deployed

## Strong candidates may also have:
- Experience building evaluations for long-horizon or agentic tasks
- Deep familiarity with Reinforcement Learning training dynamics and how model behavior changes during training
- Published research in machine learning evaluation, benchmarking, or related areas
- Experience with safety evaluation frameworks and red teaming methodologies
- Background in psychometrics, experimental psychology, or other measurement-focused disciplines
- A track record of communicating evaluation results to inform high-stakes decisions about model development or deployment
- Experience managing or mentoring researchers and engineers

## Representative projects:
- Designing and implementing a suite of long-horizon evaluations that test model capabilities on tasks requiring sustained reasoning, planning, and tool use over extended interactions
- Building systems to track capability development across RL training checkpoints, surfacing insights about when and how specific capabilities emerge
- Conducting a cross-org audit of evaluation coverage, identifying blind spots, and prioritizing new evaluations to fill critical gaps across Pretraining, RL, Inference, and Product
- Developing the evaluation methodology and narrative for a major model release, working with research leads and communications to clearly characterize model capabilities and limitations
- Researching and prototyping novel evaluation approaches for capabilities that are difficult to measure with existing benchmarks
- Leading a team effort to build reusable evaluation infrastructure that serves multiple teams across the research organization

## Similar roles

- [Research Engineer, Discovery](https://hotfix.jobs/jobs/d733a39f-1f6b-4aa7-bc66-7a2a02745fa8) - Anthropic - San Francisco, CA - $350k – $850k/yr
- [Agent Post-Training, Frontier Evals and Environments Research](https://hotfix.jobs/jobs/ff0a15ea-be79-471d-bed3-8b433b4d1f9c) - OpenAI - San Francisco, CA - $295k – $445k/yr
- [Applied Research Scientist / Engineer](https://hotfix.jobs/jobs/adbea7ef-cd6b-435e-826e-ccffeeb8cf8d) - Luma AI - New York, NY - $200k – $450k/yr
- [Senior Research Scientist](https://hotfix.jobs/jobs/2f7c7962-d3c6-4aac-87db-57b9d720858f) - EliseAI - New York, NY - $200k – $320k/yr
- [Senior Research Scientist](https://hotfix.jobs/jobs/4e9d747c-3f11-455f-bf20-0d015ffbdd3a) - Pindrop - Remote - $185k – $215k/yr

**Apply:** https://hotfix.jobs/jobs/dc095a5a-17c0-400a-9894-a77bbac72c8d
**Canonical:** https://hotfix.jobs/jobs/dc095a5a-17c0-400a-9894-a77bbac72c8d