# Research Scientist – Frontier Evaluations

**Company:** [AfterQuery](https://hotfix.jobs/companies/afterquery)
**Location:** San Francisco, CA
**Role:** AI Research
**Salary:** $210k – $450k/yr
**Skills:** Artificial Intelligence, Machine Learning, Benchmarking, Evaluation Harnesses, Statistical Methods, Experimental Design, Scientific Writing, Post-Training, Ablation Studies, Contamination Controls
**Posted:** 2026-09-10

> Designs, validates, and publishes rigorous evaluations and benchmarks for frontier AI systems across agentic, coding, safety, and expert-domain applications. The role requires strong research publication experience, scientific writing, experimental rigor, and the ability to deliver reproducible evaluation systems.

## Job Description

## Responsibilities
- Lead the end-to-end design, validation, launch, and continuous improvement of frontier AI benchmarks.
- Partner with researchers and domain experts to develop evaluations focused on model failures, coverage gaps, and high-priority domains.
- Analyze model capabilities and failure modes using rigorous experimental design and statistical methods.
- Build reproducible evaluation systems, including harnesses, graders, and benchmark infrastructure.
- Collaborate with researchers to post-train models and measure resulting performance gains.
- Communicate results through benchmark reports, technical articles, and research papers.

## Requirements
- Strong record of publishing benchmarks or research papers.
- Clear technical communication and strong scientific writing skills.
- Commitment to experimental rigor, including baselines, ablations, statistical validity, and contamination controls.
- Ability to take an ambiguous evaluation question from initial scoping through a reproducible public release.
- Depth in agentic, coding, and safety evaluations, or applied machine learning in an expert domain.

## Nice-to-haves
- PhD in a related technical field.
- Research publications at leading conferences or peer-reviewed journals.
- Interest in multidisciplinary research and the creativity to combine methods and insights from AI, engineering, science, and other expert domains.

## Compensation and Benefits
- Salary: $210,000–$450,000 annually.
- Meaningful equity.
- Medical, vision, and dental insurance.
- 401(k) with employer match.
- Daily Uber Eats stipend.
- Monthly wellness stipend.
- Commuting costs covered.

## Similar jobs

- [Research Scientist, APEX Benchmarks](https://hotfix.jobs/jobs/77bc636d-de61-47a6-9ca0-db586f4d2d22) - Mercor - San Francisco, CA - $200k – $400k/yr
- [Research Scientist](https://hotfix.jobs/jobs/f5ff3391-4477-4eb8-b26f-78c72fc10328) - Tessera Labs - San Jose, CA - $200k – $300k/yr
- [AI Engineer](https://hotfix.jobs/jobs/821a43a1-d7f0-4e9c-82eb-c41190d3bab9) - Baseten - San Francisco, CA - $220k – $260k/yr
- [People Research Scientist](https://hotfix.jobs/jobs/9e65a017-a0c0-4582-a774-59ce6f0c1e69) - OpenAI - San Francisco, CA - $198k – $220k/yr
- [AI Systems Research and Development Engineer – LLM Inference Systems & Optimization](https://hotfix.jobs/jobs/6de1a7ac-346f-4206-b1ff-39baef6d5226) - Snowflake - Bellevue, WA - $236k – $330k/yr

**Apply:** https://hotfix.jobs/jobs/7f457896-ca9c-4429-82a4-fd6190219c60
**Canonical:** https://hotfix.jobs/jobs/7f457896-ca9c-4429-82a4-fd6190219c60