# Research Scientist, Agent Robustness

**Company:** [Scale AI](https://hotfix.jobs/companies/scale-ai)
**Location:** San Francisco, CA, New York, NY
**Role:** AI Research
**Salary:** $197k – $247k/yr
**Experience:** 3+ years
**Skills:** RLHF, Dpo, Grpo, Swe-Bench, Webarena, Osworld, Inspect, Red-Teaming, Prompt Injection, Adversarial Testing, Generative AI, Machine Learning, Agent Evaluation, Rl Techniques
**Posted:** 2026-03-20

> Research Scientist focuses on agent robustness, developing tests, exploits, and mitigations for safe AI agents. Requires 3+ years ML experience, RL techniques like RLHF/DPO, and published research in generative AI.

## Job Description

## Responsibilities
- Research the science of AI agent capabilities with a focus on safety, risk factors, and benchmarking methodologies.
- Design and build harnesses to test AI agents’ tendency to take harmful actions when pressured or tricked.
- Design and build exploits and mitigations for failure modes arising from agent affordances like coding, web browsing, and computer use.
- Characterize and design mitigations for failure modes or risks in systems with multiple interacting AI agents.

## Requirements
- Commitment to promoting safe, secure, and trustworthy AI deployments.
- Practical experience conducting technical research collaboratively, including building agent scaffolding, designing evaluation harnesses, and prototyping research ideas.
- Experience with post-training and RL techniques such as **RLHF**, **DPO**, **GRPO**.
- Track record of published research in machine learning, particularly generative AI.
- At least **3 years** of experience addressing sophisticated ML problems.
- Strong written and verbal communication skills.

## Nice to Have
- Hands-on experience with agent evaluation frameworks such as **SWE-bench**, **WebArena**, **OSWorld**, **Inspect**.
- Experience with red-teaming, prompt injection, or adversarial testing of AI systems.

## Similar roles

- [Research Scientist, Frontier Risk Evaluations](https://hotfix.jobs/jobs/fff54c85-716a-40de-964d-74f8c276583e) - Scale AI - San Francisco, CA - $197k – $247k/yr
- [Research Scientist, AI Controls and Monitoring](https://hotfix.jobs/jobs/f3e18621-85f5-46d2-b41c-8ac22e105ee9) - Scale AI - San Francisco, CA - $197k – $247k/yr
- [Research Scientist](https://hotfix.jobs/jobs/bafcccc2-823d-43e6-8b59-489f23d0c2e2) - Hedra - San Francisco, CA - $200k – $325k/yr
- [AI Researcher, Core ML (Turbo)](https://hotfix.jobs/jobs/8c5b12c6-0233-4184-8ce9-9058ce02a2fe) - Together AI - San Francisco, CA - $200k – $280k/yr
- [Founding ML Researcher](https://hotfix.jobs/jobs/7b0dd9ef-0c31-4755-963e-e53665d9162e) - Unsiloed AI - San Francisco, CA - $200k – $300k/yr

**Apply:** https://hotfix.jobs/jobs/6ab91c5e-6084-4fcb-8280-cc605af029d2
**Canonical:** https://hotfix.jobs/jobs/6ab91c5e-6084-4fcb-8280-cc605af029d2