# Cyber Evaluations Engineer

**Company:** [Anthropic](https://hotfix.jobs/companies/anthropic)
**Location:** San Francisco, CA, Washington, DC
**Role:** Security Engineering
**Salary:** $300k – $405k/yr
**Skills:** Python, Cybersecurity, Security Research, Offensive Security, Exploit Development, Ai Security Benchmarks, Machine Learning, Ai/Ml Evaluation Frameworks, Jailbreak Detection, Vulnerability Research, Sigma, Yara, Suricata, SIEM
**Posted:** 2026-09-01

> Build and operate evaluations for cyber capabilities and safeguard robustness in AI models, analyze adversarial data, and develop cyber-abuse detection probes. The role requires hands-on cybersecurity experience, Python proficiency, evaluation expertise, and strong cross-functional communication.

## Job Description

## Responsibilities
- Design and run capability, uplift, and safety evaluations assessing cyber-relevant risk in new models.
- Execute safeguard-robustness testing before major model launches.
- Analyze evaluation results and communicate findings to cross-functional stakeholders.
- Design, prototype, and tune detection probes for cyber misuse.
- Collaborate with cyber policy partners to translate policy lines into layered abuse-detection architecture and measure precision and coverage over time.
- Build and maintain internal tooling for running and scoring evaluations.
- Work with policy and engineering partners to translate evaluation findings into safeguard improvements.

## Requirements
- Experience building or running evaluations, benchmarks, or test suites for software or ML systems, including delivering results on short, fixed timelines.
- Hands-on cybersecurity experience, such as CTF participation, vulnerability research, exploit development, or security research.
- Proficiency in Python.
- Strong communication skills with cross-functional and policy stakeholders.
- Bachelor's degree or equivalent combination of education, training, and experience in a relevant field.

## Nice-to-haves
- Deep offensive-security or security-research experience, including building AI security benchmarks.
- Experience analyzing adversarial or abuse data, including jailbreaks, prompt bypasses, intrusion telemetry, or fraud telemetry.
- Experience testing or evaluating systems with government partners onsite.
- Experience with AI/ML evaluation frameworks.
- Familiarity with coordinated vulnerability disclosure practices.
- Experience testing pre-release or pre-deployment software or models under confidentiality constraints.
- Experience authoring detection content such as Sigma, YARA, Suricata, or SIEM rules, or building ML-based abuse detection.
- Active secret security clearance or higher, or eligibility to obtain one.

## Compensation
- Annual salary: **$300,000–$405,000 USD**.

## Similar jobs

- [Security Engineer, Offensive Security](https://hotfix.jobs/jobs/e5c4fc22-2c3a-4334-8eae-e8ab5bcf2a8a) - Anthropic - San Francisco, CA - $300k – $320k/yr
- [Security Engineer - Threat Intel](https://hotfix.jobs/jobs/6755c877-3974-4901-9d13-a4c8d6591236) - Anthropic - New York, NY - $320k – $405k/yr
- [Security Engineer, Corporate Security](https://hotfix.jobs/jobs/5bfe00e1-07b2-44ee-b927-2586d89c7fe7) - Anthropic - San Francisco, CA - $320k – $405k/yr
- [Software Engineer, HSM Infrastructure Security, Consumer Devices](https://hotfix.jobs/jobs/c59b2911-dd77-45cf-a787-90da0f7be1b7) - OpenAI - San Francisco, CA - $347k – $445k/yr
- [Security Engineer, Threat Intelligence](https://hotfix.jobs/jobs/a9d333c0-2242-416f-a470-d3a19f982046) - Fluidstack - New York, NY - $220k – $280k/yr

**Apply:** https://hotfix.jobs/jobs/5720917b-cc75-4d96-9629-7c3fc9a5d6cd
**Canonical:** https://hotfix.jobs/jobs/5720917b-cc75-4d96-9629-7c3fc9a5d6cd