Skip to content
AfterQueryAfterQuerySan Francisco, CA

Research Scientist - Frontier Data

Designs datasets and evaluation frameworks for frontier AI models, collaborating with top labs to expose failure modes, refine RLHF/RLVR pipelines, and measure data impact on capabilities. Requires strong quantitative skills, familiarity with LLM training, and research experience up to master's level.

150k – 250k
On-siteAI Research

About the role

What You'll Do

  • Design data slices and explore data shapes that expose meaningful model failure modes across domains like finance, code, and enterprise workflows
  • Build and refine evaluation rubrics and reward signals for RLHF and RLVR training pipelines
  • Model annotator behavior and run experiments to improve different model capabilities
  • Develop quantitative frameworks for measuring dataset quality, diversity, and downstream impact on model alignment and capability
  • Partner with lab research teams to translate their training objectives into concrete data and evaluation specifications

What We're Looking For

  • Great candidates are undergrad research or master's research (but haven't done a PhD)
  • Major plus if they've worked for/interned for any RL environment companies in the past or any AI safety or benchmarking orgs like METR, Artificial Analysis, etc.
  • Genuine obsession with how data structure, selection, and quality drive model behavior
  • Ability to design lightweight experiments, move fast, and extract actionable insights from messy results
  • Comfort working across domains (you'll touch finance, software engineering, policy, and more)
  • Strong quantitative instincts and familiarity with LLM training pipelines, RLHF/RLVR, or evaluation methodology
  • A bias toward building over theorizing

Compensation Structure

$250k-450k total compensation + equity

Skills

RLHFRlvrLLMsData CollectionEvaluation FrameworksDataset QualityModel AlignmentQuantitative AnalysisExperiment DesignAi Safety

Similar roles

AI Research jobs
Distyl AI

Applied AI Researcher, Multi-Agent Systems

Distyl AISan Francisco, CA +1

Develop multi-agent AI architectures for enterprise coordination and collaborative reasoning. Requires research experience in MARL/GNNs, strong prototyping skills, and daily AI tool usage.

150k – 250k
HybridAI Research
Basis

Intelligence Architect

BasisNew York, NY

Designs and authors context, procedures, skills, and system prompts for AI agents performing autonomous accounting tasks. Ensures consistency, monitors performance, and shapes future capabilities through precise language and systems thinking.

150k – 225k
On-siteAI Research
AfterQuery

Research Scientist - Post Training

AfterQuerySan Francisco, CA

Design and run SFT/RL experiments to measure dataset impact on LLM performance, capabilities, and alignment. Collaborate with labs to provide evidence of improvements; requires strong LLM training knowledge and fast experimentation, ideally undergrad/master's research background.

150k – 250k
On-siteAI Research
Astera

Machine Learning Researcher - Springtail

AsteraEmeryville, CA

Conducts fundamental research on data-efficient ML architectures, including bootstrapped program synthesis and self-synthesizing learning systems. Requires Master's in ML/math, PyTorch fluency, and research experience.

150k – 300k
HybridAI Research
Astera

Research Scientist: Hierarchical Sensorimotor Perception

AsteraEmeryville, CA

Develops bi-directional/recurrent architectures for hierarchical sensorimotor perception inspired by brain principles to advance AGI. Requires PhD-level expertise in deep learning frameworks like PyTorch/JAX, theoretical rigor, and passion for biological intelligence.

150k – 300k
HybridAI Research