Red Team Specialist - Cyber
The Red Team Specialist evaluates AI models for cyber capabilities, safeguard failures, and agentic-system abuse risks. The role combines hands-on security testing, automated evaluation infrastructure, risk assessment, and cross-functional communication.
About the job
Responsibilities
- Design and run rigorous evaluations of model cyber capabilities and safeguards, including policy adherence, correct refusal, over-refusal, and resilience to jailbreaking and other adversarial techniques.
- Conduct hands-on testing to understand what models can enable when used by experienced security practitioners, including through task-specific harnesses, scaffolding, and multi-step workflows.
- Distinguish benchmark or policy failures from behavior that creates meaningful real-world risk, considering feasibility, attacker uplift, reliability, and capabilities already available elsewhere.
- Build and improve automated testing infrastructure for repeatable measurement, rapid iteration, and statistically grounded analysis across models and product surfaces.
- Test novel abuse risks in agentic systems, including indirect prompt injection, agent hijacking, and other adversarial manipulation of tool-using systems.
- Translate findings into clear risk assessments and actionable recommendations for Security, Research, Product, Policy, and Engineering partners.
- Contribute to Safety Bug Bounty work, particularly reports requiring cyber expertise.
Requirements
- Substantial depth in at least one of the following areas:
- Cybersecurity, including application security, penetration testing, vulnerability research, adversary simulation, or red-team operations.
- AI model evaluation, including designing and running evaluations, building agentic harnesses, automating adversarial testing, constructing datasets, or analyzing model behavior at scale.
- Working literacy across both cybersecurity and model evaluation, with interest in developing further depth outside the primary area.
- Ability to write code and build practical testing tools, particularly for automating experiments, orchestrating models, or analyzing results.
- An attacker mindset and interest in discovering failure modes that standard evaluations may not capture.
- Clear written and verbal communication, including the ability to explain technical findings, limitations, and risk to audiences with different backgrounds.
- Experience working across technical and non-technical teams to move from a finding to a decision, mitigation, or follow-up test.
Compensation
- Salary range: $198,000–$320,000.
Skills
Cybersecurity, Application Security, Penetration Testing, Vulnerability Research, Adversary Simulation, Red Teaming, Python, Ai Model Evaluation, Agentic Systems, Prompt Injection, Jailbreaking, Automation, Statistical Analysis, Security Testing, Risk Assessment
Similar jobs
Security Engineering jobs
Vercel
San Francisco, CABuild and operate trust and safety systems that detect and mitigate abuse at internet scale. The role combines security engineering, large-scale data analysis, and applied LLM techniques, requiring 5+ years of relevant experience and strong Python and JavaScript/TypeScript skills.
1Password
United StatesLeads and develops a security incident response team while driving automation, AI-assisted workflows, operational maturity, and response strategy. The role requires 5+ years of incident response experience, people leadership, technical depth, and calm management of high-severity incidents.
Decagon
San Francisco, CA
Own and operate Decagon’s privacy and GRC programs, including regulatory compliance, data governance, customer security engagements, audits, and cross-functional privacy initiatives. The role requires 5+ years of GRC experience, strong communication and project management skills, and familiarity with enterprise security controls.
Anthropic
San Francisco, CAThe analyst develops and evaluates cyber product policies, enforcement guidance, controlled-access frameworks, and launch-review inputs for AI systems. The role requires strong policy writing, cybersecurity or platform-enforcement familiarity, technical security literacy, and cross-functional communication.
Decagon
San Francisco, CA
Manages Decagon’s governance, risk, and compliance program, including enterprise certifications, audit evidence, vendor risk, customer security assessments, and RFP responses. Requires 3–5 years of GRC experience, strong communication and project management skills, and familiarity with technical security controls and privacy regulations.