Design Program Manager, AI Evals
Owns contractor, vendor, and operational programs supporting AI-generated design evaluations. The role requires design craft expertise, strong UI judgment, and at least three years of design program management or adjacent experience.
About the job
Responsibilities
- Drive quality and velocity across the design evaluation pipeline by running annotation rhythms, improving calibration and judgment standards, and connecting evaluation output to product decisions.
- Define high-quality reference designs that establish reliable benchmarks for AI-generated design output.
- Own contractor hiring end to end, including sourcing, screening, closing, and scaling headcount based on evaluation demand.
- Manage vendor logistics, budgets, statements of work, contracts, and purchasing in partnership with Legal, Security, and Procurement.
- Run day-to-day program management for contractors and evaluation workstreams, including status, blockers, timelines, and dependencies.
- Partner with model designers to review contractor evaluation output and maintain quality standards.
- Identify opportunities to automate or eliminate operational overhead across evaluation workstreams, including through internal tooling.
Requirements
- 3+ years of experience in design program management, DesignOps, or an adjacent design role.
- Deep design craft expertise, including strong UI judgment and the ability to coach others on design quality.
- Experience procuring, onboarding, and managing design contractors or third-party vendors.
- Experience driving operational programs that improve team processes through rituals, documentation, and workflow improvements.
- Strong project management and written communication skills.
- Experience coordinating contractors and vendors across multiple time zones.
Nice-to-haves
- Hands-on experience with Figma features such as auto layout, variables, components, and Slides.
- Experience orchestrating AI tools, APIs, or workflow automations to simplify program management.
- Experience running or participating in design evaluation or quality review programs.
Skills
Design Program Management, Designops, UI Design, Project Management, Vendor Management, Contractor Management, Annotation, Quality Assurance, Figma, Auto Layout, Design Systems, APIs, Workflow Automation, AI Tools, Procurement
Similar jobs
Technical Program Management jobs
Encord
London, United Kingdom
Own end-to-end delivery of complex human data and annotation programs for AI customers, translating ML requirements into scalable workflows while managing quality, throughput, timelines, and distributed annotator performance. Requires 3–7 years of operational and analytical experience plus Python or SQL proficiency.
Cohere
New York, NYThe People Project Manager will drive global HR programs and initiatives, building plans, governance, workflows, and accountability across cross-functional teams. The role requires at least three years of project or program management experience, strong organization, communication, and stakeholder management skills.
Stripe
Dublin, IrelandOwn and scale Stripe’s core privacy operations programs, documentation, workflows, and request intake while partnering with Privacy Legal and Engineering. Requires 5+ years of privacy program management experience and expertise in global privacy regulations and operational frameworks.
Prompt Health
Remote
Own the roadmap for healthcare APIs, EHR integrations, interoperability initiatives, and the Patient Portal while coordinating product, engineering, go-to-market teams, and external partners. Requires 3–5 years of healthcare integration experience, including FHIR, HL7, major EHRs, HIPAA, and public APIs.
Celonis
London, United Kingdom
Owns the strategy, execution, measurement, and optimization of cross-functional go-to-market programs across Sales, Marketing, Customer Success, and Product. The role requires strong program management, stakeholder alignment, SaaS GTM knowledge, analytical ability, and practical interest in AI technologies.