Skip to content
hudhudSan Francisco, CA

Research Engineer, QC Automation

Build QC automation systems for RL training data and agent evals at HUDHUD. Design quality standards, validation pipelines, experiments and metrics without heavy LLM reliance; partner with vendors to debug and improve data generation. Requires Python, Docker, Linux and experience building scalable QA/QC systems end-to-end.

Salary not listed
On-siteML Engineering

About the role

Responsibilities

  • Create QC systems based on true understanding and human judgement, without relying heavily on LLMs
  • Define and enforce quality standards for training data
  • Design experiments and metrics to grade agent outputs
  • Partner with data vendors to debug quality issues and diagnose agent failure modes, provide actionable feedback, and improve their data generation processes
  • Translate QC learnings into systems for auditing supplier-generated datasets, including sampling strategies, validation pipelines (rule-based and model-assisted), and feedback loops
  • Continuously integrate QC learnings into infrastructure tools and data vendor portal to reduce anomalies, inconsistencies, and edge cases

Requirements

  • Proficiency in Python, Docker, and Linux environments
  • Strong understanding of what “good data” means and how to measure it
  • Built scalable data validation pipelines and automated QA/QC systems end-to-end without a fully prescribed roadmap
  • Experience working on benchmarks and evals - you can reason about what makes a task realistic, a rubric reliable, an environment usable, and a trajectory useful for RL training
  • Early-stage startup experience with ability to work independently in fast-paced environments

Nice-to-Haves

  • Detail-oriented and able to spot subtle inconsistencies or edge cases in data
  • Comfortable designing metrics, experiments, and QA/QC processes, not just executing them
  • Experience with existing benchmarks and can reason about how to construct tasks in new evals
  • Thrive in unstructured problem spaces
  • Strong communication skills for remote collaboration across time zones

We prioritize technical aptitude and learning potential over years of experience. Motivated candidates are encouraged to apply even if they don't meet all criteria.

Compensation & Benefits

  • Competitive compensation based on experience and location
  • 100% covered top-of-the-line medical, dental, and vision from Blue Shield of CA
  • Lunch and dinner when you’re in the office
  • Company-wide holiday break (Christmas Eve to New Year’s Day) on top of PTO and paid holidays
  • Other perks including an Equinox membership, 401k, and commuter benefits
  • Unlimited* access to tokens for ChatGPT, Claude Code, Cursor, etc.

Skills

PythonDockerLinuxData ValidationQa/Qc SystemsBenchmarksEvalsMetrics DesignRl TrainingSampling StrategiesValidation Pipelines

Similar roles

ML Engineering jobs
Cerebras Systems

CoDesign & NextGen Performance Engineer

Cerebras SystemsSunnyvale, CA

Characterize, analyze, and optimize performance of state-of-the-art AI models on Cerebras' wafer-scale hardware. Build performance models, optimize kernels and compilers, debug runtime behavior, and develop visualization tools to influence next-gen AI architecture.

Salary not listed
On-site3+ YOEML Engineering
OpenAI

Research Engineer, Privacy

OpenAISan Francisco, CA

Research Engineer on OpenAI's Privacy team designing and prototyping privacy-preserving ML algorithms like differential privacy and federated learning at scale. Requires hands-on PETs experience, fluency in PyTorch/JAX, and a track record implementing or publishing novel privacy work.

380k – 445k
HybridML Engineering
Console

Research Engineer

ConsoleSan Francisco, CA

Research Engineer building self-improving AI agent systems at Console. Develop eval/optimization loops, fine-tune specialist models, and improve agent reasoning over enterprise context using production data to drive measurable gains in quality, latency, and reliability.

200k – 350k
On-siteML Engineering
Notion

Software Engineer, AI Platform

NotionSan Francisco, CA +1

Build and scale the shared AI platform foundations at Notion, enabling fast and safe shipping of AI products. Requires experience with LLM/ML platforms, strong ownership, and comfort across backend, infrastructure, and product code.

180k – 201k
Hybrid5+ YOEML Engineering
Liftoff

Machine Learning Engineer

LiftoffCalifornia

Machine Learning Engineer building statistical models, optimization systems, and experiments for mobile ad tech economics on the Revenue Engine team. Requires PhD in CS/ML/Economics and industry experience applying ML or economics at scale.

215k – 275k
RemoteML Engineering