Quality Control Lead
Leads final quality review for technical and AI training datasets, improves QC automation and processes, and reports quality metrics to leadership. Requires 3+ years in a technical role, programming proficiency, rigorous evaluation skills, and experience leading QA/QC functions.
About the job
Responsibilities
- Own the final quality checkpoint before datasets are delivered or published.
- Audit code, technical data, reasoning data, and knowledge-work data for correctness, clarity, structure, and adherence to specifications.
- Work with infrastructure engineers to automate and improve quality-control workflows.
- Partner with data delivery teams to identify systemic quality issues, provide feedback for contributor training, and prevent recurrence.
- Own quality metrics and KPIs, including error rates, root causes, and improvement trends, and report them to company leadership.
- Support company-wide initiatives involving building, analysis, coordination, and execution.
Requirements
- 3+ years of experience in a technical role such as software engineering, data science, or machine learning engineering.
- Hands-on proficiency in at least one modern programming language.
- Ability to review, debug, and evaluate code, technical data, and knowledge-work data for correctness, structure, and quality.
- Experience building or leading a quality assurance or quality control function for a technical or data-centric product.
- Exceptional attention to detail and sound judgment when evaluating ambiguous or contested technical content.
- Strong leadership and communication skills, including the ability to translate technical quality issues into actionable feedback.
- Ability to work independently in ambiguous, fast-moving environments.
- Passion for artificial intelligence and an entrepreneurial mindset.
- Demonstrated competitive success.
Nice-to-haves
- Experience with AI training data, RLHF, or data-annotation quality assurance.
- Experience reviewing datasets across code, reasoning, and knowledge-work domains.
- Experience designing rubrics, grading criteria, or automated quality-check tooling and scripts.
- Familiarity with Claude Code or similar AI-assisted workflow tools.
- Computer science degree or equivalent hands-on engineering experience.
Compensation and benefits
- Salary: $150,000-$275,000 annually.
- Medical, vision, and dental insurance.
- 401(k) with employer match.
- Daily UberEats stipend.
- Monthly wellness stipend.
- Commute coverage.
Skills
Quality Control, Quality Assurance, Python, Machine Learning, Data Science, Software Engineering, RLHF, Data Annotation, Dataset Evaluation, Debugging, Kpi Reporting, Claude Code
Similar jobs
QA Engineering jobs
VGS
San Francisco, CA
Senior software engineer responsible for building automated quality strategies, high-concurrency performance tests, and emulators for mission-critical payment systems. The role requires 5+ years of quality engineering experience and hands-on expertise with automation, cloud-native CI/CD, distributed systems, and observability.
Applied Intuition
Sunnyvale, CA
The Senior Validation Engineer develops and executes automotive system and integration tests using HIL benches, vehicles, and automated Python tooling. The role requires a master’s degree in an engineering discipline and at least one year of automotive test engineering experience.
Circle
Remote
Senior Quality Engineer partnering across product teams to assess risk, build web and API test automation, investigate technical failures, and improve quality practices. Requires strong Playwright and TypeScript experience, exploratory testing skills, CI/CD knowledge, and the ability to debug application code.
Vestmark
Boston, MA
The Senior SDET designs and maintains automated tests, strengthens CI/CD quality pipelines, and applies AI-assisted testing strategies across Agile software delivery. The role requires 7+ years of QA experience, strong Python and Pytest skills, Selenium WebDriver expertise, and familiarity with APIs, SQL, containers, and CI tooling.
Fora
New York, NY
The Senior QA Engineer validates backend integrations, APIs, data flows, and end-to-end user experiences while investigating complex cross-layer failures. The role requires 7+ years of QA experience, strong SQL and API testing skills, and proficiency with observability and debugging tools.