Product Manager, Claude Code Model Performance

Responsibilities

• Own model launch planning and execution for Claude Code: define readiness criteria, coordinate across research and product engineering, and ensure launches land cleanly with developers
• Design and implement agentic evals that measure real-world coding performance
• Drive the engineering team's eval roadmap
• Partner with researchers working on coding capabilities to define target behaviors and influence model development with evidence from real usage
• Talk with users and analyze transcripts to understand capability gaps and turn research progress into shipped improvements
• Synthesize signal from internal users, external developers, and competitive benchmarks into clear priorities

Requirements

• Have personally built agentic evals (e.g. SWE-bench-style task suites)
• Are a daily Claude Code user and can articulate what behaviors you’d want to change or add to the model
• Have an engineering background and 2+ years in product management, or equivalent experience driving product direction as an engineer
• Have a deep grasp of AI concepts and are comfortable going deep on model behavior, prompt engineering, and evaluation methodology
• Are a systems thinker: when you find a problem, you build the infrastructure that prevents its whole class
• Have launched products or capabilities in ambiguous, research-adjacent environments
• Have a creative, hacker spirit and love solving puzzles

Nice-to-Haves

• Minimum education: Bachelor’s degree or an equivalent combination of education, training, and/or experience
• Required field of study: A field relevant to the role as demonstrated through coursework, training, or professional experience