Human Data Engineer

Responsibilities

• Partner with model and engineering teams to understand needs and translate them into high-value data projects and evaluation strategies.
• Own end-to-end delivery of critical data and evaluation projects that capture meaningful training signals and support rapid model development.
• Build tools and systems to measure data effectiveness, including data yield, evaluation lift, and usage; run evaluations to continuously improve quality.
• Maintain rigorous data integrity and truthfulness, including validation processes for factual accuracy; prioritize quality over quantity.
• Research, implement, and evaluate techniques for data collection, annotation, generation, and multimodal integration.
• Shape model behavior and domain performance through targeted data work.
• Design and improve annotation workflows, labeling interfaces, and related tools with a focus on data quality and integrity.
• Act as a liaison between engineering, technical staff, and Human Data teams to drive alignment and knowledge sharing.
• Collaborate with Human Data Operations and stakeholders to scale projects, share learnings, contribute to demand forecasting, and report status, insights, and blockers.

Requirements

• Bachelor's degree in engineering, computer science, or a related STEM discipline, or 4+ years of experience in lieu of a degree.
• Experience collaborating with cross-functional teams, including engineering, research, product, or annotation and operations groups.
• Demonstrated experience analyzing datasets to identify trends, anomalies, quality issues, or integrity problems.

Preferred Qualifications

• Master's degree or higher in a relevant technical field.
• Direct experience curating, evaluating, or improving training or evaluation datasets for large language models or other AI/ML systems.
• Experience designing, supporting, or optimizing annotation tools, labeling interfaces, or data workflows that prioritize factual accuracy and data integrity.
• Familiarity with multimodal data, including text, images, code, or other modalities, or with domain-specific data.
• Experience conducting model or dataset evaluations focused on quality, truthfulness, or alignment with product goals.
• Experience with a scripting language such as Python.
• Experience with SQL or other data analysis tools.

Additional Requirements

• Weekend work may be required.
• Travel to other company sites may be required.

Compensation and Benefits

• Base salary: $144,000–$270,000 USD.
• Equity, comprehensive medical, vision, and dental coverage.
• Access to a 401(k) retirement plan.
• Short- and long-term disability insurance.
• Life insurance, discounts, and other perks.