Skip to content
FigmaFigmaSan Francisco, CA

Director, Research

Lead AI evaluation for Figma's AI-powered products. Define quality metrics, build human + automated eval frameworks (rubrics, golden datasets, LLM-as-judge), manage a small team, and deliver decision-ready insights to Product, Design, and Engineering stakeholders.

258k – 348k
Hybrid10+ YOEAI Research

About the role

What you'll do at Figma

  • Own AI evaluation methods and operations for Figma's AI-powered experiences — define quality dimensions, design how we measure them, and turn results into decision-ready signal
  • Build and maintain evaluation frameworks, rubrics, golden datasets, and quality bars, combining human evaluation with automated/model-based approaches (e.g., LLM-as-judge) where appropriate
  • Partner with engineering to stand up repeatable, reproducible evaluation pipelines and regression testing, so evaluation is a routine part of how AI features are built and shipped
  • Produce clear readouts and dashboards that let stakeholders confidently make go/no-go and prioritization decisions
  • Socialize a shared definition of quality so evaluation standards are adopted across teams rather than re-invented — and advocate for evaluation as a strategic partner in the product process
  • Manage a small team to execute our AI evals in partnership with contractors, internal staff, and/or LLMs

Requirements

  • 10+ years of experience in product, research, applied research, or a closely related field, including 2+ years of management experience
  • Direct, hands-on experience owning the evaluation of AI/LLM-powered products
  • Expertise designing and running AI evaluation — human evaluation programs, rubric and benchmark/golden-dataset construction, inter-rater reliability — and sound judgment about when and how to apply automated/model-based approaches (e.g., LLM-as-judge), including their limitations
  • Strength across both qualitative and quantitative methods, comfort with data and metrics, and the ability to reason about model behavior
  • Demonstrated success in identifying the riskiest assumptions behind an ambiguous quality question, prioritizing them, and designing right-sized evaluation to build confidence
  • A proven track record of gaining buy-in from executive and cross-disciplinary stakeholders — transcending methodology to articulate a larger user story and the "so what" to inspire action

Nice-to-haves

  • Experience building or co-building automated evaluation pipelines and regression testing in partnership with engineering, or familiarity with eval tooling (e.g., Braintrust, LangSmith, DeepEval, or equivalents)
  • Experience standing up a new function, practice, or discipline from scratch
  • 2+ years in product design, user-centric product management, data science, product development, and/or front-end engineering
  • A familiarity and depth of experience using Figma's products

Skills

Ai EvaluationLlm EvaluationHuman EvaluationRubric DesignGolden DatasetsLlm-As-JudgeInter-Rater ReliabilityQualitative MethodsQuantitative MethodsEvaluation PipelinesRegression TestingBraintrustLangsmithDeepeval

Similar roles

AI Research jobs
Scale AI

Head of Policy & Security Research Lab

Scale AISan Francisco, CA +2

Lead Scale’s Policy Research Lab, setting strategy and managing research on AI safety, benchmarks, and frontier risk evaluations while collaborating with government institutes and academia. Requires 6+ years experience, strong technical/policy background, and proven team leadership.

198k – 247k
On-site6+ YOEAI Research
Kepler

Head of AI Research

KeplerNew York, NY

Lead AI research defining the agenda for trustworthy agentic systems, memory, retrieval, and evaluation frameworks. PhD required with 5+ years focused on LLMs/agents and a strong publication record; work ships directly to production for high-stakes financial users.

400k – 700k
On-site5+ YOEAI Research
Invisible Tech

Head of AI Training Research

Invisible TechUnited States

Lead applied AI research practice delivering client outcomes through benchmarking, data strategy, and RL environments. Drive revenue impact, scope engagements, and lead cross-functional teams of researchers and engineers.

Salary not listed
Remote8+ YOEAI Research
Protege

Director of Research, DataLab

ProtegeUnited States

Lead Protege's DataLab research organization, defining strategy for AI training data quality, evaluation systems, and marketplace optimization while managing researchers and partnering with Product, Engineering, and GTM.

Salary not listed
RemoteAI Research
Databricks

Principal Research Scientist - AI Scaling & Optimization

DatabricksMountain View, CA +1

Leads research team advancing LLM scaling, efficiency, post-training, RL, and inference optimization. Drives innovations from research to production, requiring deep ML expertise, Python/PyTorch proficiency, and leadership in large-scale experiments.

270k – 350k
On-siteAI Research