Human Data Manager

Responsibilities

• Partner with model and engineering teams to understand needs and translate them into high-value data projects and evaluation strategies.
• Own end-to-end delivery of critical data and evaluation projects that capture meaningful training signals and support rapid model development.
• Leverage AI agents and existing platforms to measure data effectiveness and quantify impact, including data yield, evaluation lift, and usage.
• Maintain rigorous data integrity and truthfulness, including validation processes for factual accuracy; prioritize quality over quantity.
• Research and apply techniques for data collection, annotation, generation, and multimodal integration.
• Shape model behavior and domain performance through targeted data work; improve annotation workflows using agents and no/low-code approaches where helpful.
• Manage plans that shape model behavior via data management, optimization, and analysis, including resources and timelines.
• Act as a liaison between engineering, technical staff, and tutoring or Human Data teams to drive alignment and knowledge sharing.
• Collaborate with Human Data Operations and stakeholders to scale projects, share learnings, contribute to demand forecasting, and report status, insights, and blockers.

Requirements

• Bachelor’s degree, or 4+ years of relevant experience in lieu of a degree.
• Experience collaborating with cross-functional teams, including engineering, research, product, or annotation and operations groups.
• Demonstrated experience analyzing datasets to identify trends, anomalies, quality issues, or integrity problems.

Nice-to-haves

• Degree in engineering, computer science, data analysis, or a related STEM discipline.
• Experience curating, evaluating, or improving training or evaluation datasets for large language models or other AI/ML systems.
• Experience designing, supporting, or optimizing annotation workflows or data processes that prioritize factual accuracy and data integrity.
• Familiarity with multimodal data or domain-specific data such as science, mathematics, programming, or recent events.
• Experience with model or dataset evaluations focused on quality, truthfulness, or alignment with product goals.
• Comfort using AI agents, no/low-code tools, or light scripting such as Python to prototype workflows and measurement.
• Experience with SQL or other data analysis tools.

Additional Requirements

• Weekend work may be required.
• Travel to other company sites may be required.

Compensation and Benefits

• Base salary: $100,000–$186,000 USD.
• Equity and comprehensive medical, vision, and dental coverage.
• 401(k) retirement plan.
• Short- and long-term disability insurance.
• Life insurance, discounts, and other perks.