Researcher, Frontier Cybersecurity Risks
Design and deploy scalable safeguards that mitigate cybersecurity misuse by frontier AI models across OpenAI product surfaces. The role requires deep learning and transformer expertise, software engineering fundamentals, LLM fine-tuning experience, and cross-functional collaboration.
About the job
Responsibilities
- Design and implement mitigation components for model-enabled cybersecurity misuse, spanning prevention, monitoring, detection, and enforcement.
- Integrate safeguards across product surfaces with product and engineering teams, ensuring protections are consistent, low-latency, and scalable.
- Evaluate technical trade-offs involving coverage, latency, model utility, and user privacy, and propose pragmatic, testable solutions.
- Collaborate with risk and threat-modeling partners to align mitigation design with anticipated attacker behaviors and high-impact misuse scenarios.
- Execute rigorous testing and red-teaming workflows against evolving threats, including novel exploits, tool-use chains, and automated attack workflows.
- Iterate on the mitigation stack based on testing findings across different product surfaces.
Requirements
- Passion for AI safety and making cutting-edge AI models safer for real-world use.
- Demonstrated experience with deep learning and transformer models.
- Proficiency with PyTorch or TensorFlow.
- Strong foundation in data structures, algorithms, and software engineering principles.
- Familiarity with training and fine-tuning large language models, including distillation, supervised fine-tuning, and policy optimization.
- Ability to collaborate across research, security, policy, product, and engineering teams.
- Significant experience designing and deploying technical safeguards for abuse prevention, detection, and enforcement at scale.
Nice to Have
- Background knowledge in cybersecurity or adjacent fields.
Skills
Deep Learning, Transformer Models, PyTorch, TensorFlow, Data Structures, Algorithms, Software Engineering, LLMs, Knowledge Distillation, Supervised Fine-Tuning, Policy Optimization, Red Teaming, Threat Modeling, Cybersecurity, Abuse Prevention
Similar jobs
ML Engineering jobs
OpenAI
San Francisco, CA
Build and optimize OpenAI’s inference stack for AWS Trainium across high-performance kernels, compilers, runtimes, and model execution. The role requires systems programming and accelerator experience, with opportunities to solve end-to-end performance problems for frontier-scale AI models.
Anthropic
San Francisco, CABuild and deploy LLM-powered tools, agents, and ecosystem infrastructure with life sciences research institutions. The role requires deep scientific or biomedical research experience, production software development expertise, and the ability to translate partner workflows into scalable AI systems.
OpenAI
San Francisco, CA
Build research infrastructure and tooling that enables AI models to design silicon, including reinforcement learning environments, EDA integrations, evaluations, and experiment workflows. The role requires strong software engineering fundamentals and comfort working across research, tooling, and chip-design systems.
OpenAI
San Francisco, CA
Build and optimize the production LLM inference runtime for frontier models on OpenAI’s custom silicon. The role spans scheduling, distributed execution, memory and KV-cache management, performance tooling, and hardware-software co-design.
Hyperbound
San Francisco, CA
Build and operate machine learning models for sales roleplay, scoring, and coaching products, owning the lifecycle from fine-tuning and evaluation through production and on-device deployment. The role emphasizes open-source models, latency and privacy optimization, and rigorous model testing.