Research Engineer, Takeoff Intel
Research Engineer building large-scale AI capability evaluations, telemetry, data pipelines, and analysis tools for Anthropic’s Takeoff Intel team. The role requires hands-on large language model experimentation, rapid prototyping, data expertise, and strong research collaboration.
About the job
Responsibilities
- Design, build, and run capability evaluations and measurement instruments at scale.
- Build data and analysis pipelines that transform large volumes of model outputs and telemetry into reliable metrics.
- Prototype new instruments rapidly, validate them, and determine which to retain.
- Review and supervise AI-written code.
- Collaborate with research scientists and partner teams to define meaningful measurements.
- Contribute to internal write-ups and public reporting.
Requirements
- End-to-end experience shipping an evaluation, data product, or research library.
- Ability to prototype quickly and discard code when appropriate.
- Experience handling messy, large-volume data without over-engineering.
- Experience running experiments on large language models.
- Ability to work from ambiguous research questions rather than detailed specifications.
- Strong written and verbal communication and close collaboration skills.
Nice-to-haves
- Experience building LLM evaluation harnesses or benchmark infrastructure.
- Experience with large-scale machine learning or data infrastructure, including self-driving or observability systems.
- Experience building tools or libraries used by other researchers.
- Experience identifying errors in AI-written code.
Compensation
- Annual salary: $350,000–$850,000 USD.
Skills
LLMs, Machine Learning, Data Pipelines, Data Processing, Evaluation Harnesses, Benchmark Infrastructure, Telemetry, Python, Research Libraries
Similar jobs
AI Research jobs
Thinking Machines Lab
San Francisco, CA
Research role focused on improving agentic coding capabilities through reinforcement-learning training, synthetic data, coding environments, reward design, and evaluations. Requires strong Python engineering, scalable distributed-training experience, and a bachelor’s degree or equivalent; research experience and a PhD are preferred.
Thinking Machines Lab
San Francisco, CA
Conduct AI safety research across data curation, post-training, evaluations, synthetic data, and red-teaming to improve model reliability on harmful and dual-use requests. The role requires AI safety experience, Python, deep learning frameworks, and scalable technical research skills.
OpenAI
San Francisco, CA
Researcher or engineer focused on designing, evaluating, and productionizing oversight systems and safety mitigations for autonomous AI agents. The role requires strong systems or security reasoning, threat-modeling ability, and experience building practical evaluations and controls.
OpenAI
San Francisco, CA
Researcher focused on training and evaluating frontier AI agents, mining incidents, and building scalable safety measurement systems. The role requires strong research or ML engineering execution, quantitative judgment, and the ability to own ambiguous projects end to end.
Anthropic
San Francisco, CA
Conducts hands-on medicinal chemistry research to evaluate AI-generated molecules and synthetic routes, advancing small-molecule programs from design through experimental validation. Requires a chemistry PhD, sustained synthetic experience, and cross-functional collaboration skills.