# Research Operations, Code

**Company:** [Mercor](https://hotfix.jobs/companies/mercor)
**Location:** San Francisco, CA
**Role:** Data Engineering
**Salary:** $130k – $250k/yr
**Experience:** 5+ years
**Skills:** Python, Machine Learning, LLMs, Model Evaluation, benchmarking, Data Pipelines, synthetic data, model-in-the-loop systems, Project Management, Technical Communication
**Posted:** 2026-07-22

> Own end-to-end multi-million-dollar code data programs for frontier AI labs. Decompose model weaknesses, design human/synthetic/hybrid data pipelines, manage expert teams, and serve as the trusted technical partner to researchers. Requires strong coding literacy, ML familiarity, and operational ownership to deliver high-quality data under tight timelines.

## Job Description

## Core Responsibilities

### Frontier Model Evaluation & Data Strategy
- Develop a working understanding of what our customers' models can and can't do, and where the capability gaps are.
- Evaluate model outputs and benchmark performance, and create targeted loss analysis to identify what data will drive improvement.
- Translate research goals into concrete task designs, difficulty targets, and quality specifications — including novel frontier tasks that don't exist anywhere else.

### Pipeline Design & Innovation
- Design how data is produced — human-expert, synthetic, and hybrid model-in-the-loop pipelines — and continuously improve them.
- Prototype new generation and validation approaches; bring ideas for new data products, not just improvements to existing ones.
- Balance quality, throughput, and cost as you scale a pipeline from prototype to production.

### Operational Excellence
- Manage end-to-end data pipelines from customer specification to final delivery.
- Diagnose bottlenecks, restructure workflows, and implement solutions — incentive systems, workflow re-sequencing, sharper instructions, scaled review processes, and automated quality assurance.
- Run daily internal syncs ("war rooms") to stay ahead of issues.

### Customer Relationships
- Act as the primary point of contact for leading AI labs; deliver clear, consistent reporting.
- Proactively anticipate researcher needs and identify opportunities for expansion.

### Expert Teams & Workflow Design
- Design the human-expert labeling and evaluation flows on the platform — how tasks are structured, reviewed, and scored.
- Source, vet, train, and performance-manage teams of domain experts (software engineers, competitive programmers, and specialists).
- Maintain a high execution and quality standard across every stage of production.

## Requirements
- **Technical judgment**: Coding literacy and genuine ML or Model Benchmark familiarity — enough to evaluate model outputs, reason about frontier-model capabilities and failure modes, read benchmark/eval work, and translate research goals into concrete task and data designs.
- **Operational ownership**: A track record of running complex, high-stakes projects end to end, and genuine energy for large-scale execution and gritty process optimization under pressure.
- **Communication & customer instinct**: Strong analytical and communication skills; comfortable owning high-profile relationships with technical customers.

## Nice-to-Haves
- Pipeline building: designing data or automation pipelines; hands-on with LLMs/agents; building synthetic-data or model-in-the-loop systems.
- Research fluency: connecting model/benchmark literature to what data would move a frontier model; having created a benchmark or published analysis of model behavior.

Backgrounds that often fit: ML/data/software engineers who love operating, technical PMs, research engineers, or strong generalist operators with real technical range. Backgrounds from consulting, finance, or high-growth startups can work if paired with real technical fluency.

## Compensation & Benefits
- Bi-annual performance bonus structure
- Generous equity grant vested over 4 years
- Up to $15k Relocation bonus
- $10K housing bonus (if you live within 0.5 miles of our office)
- $1.5K monthly stipend for meals
- Free Equinox membership
- $200 monthly laundry reimbursement
- $200 monthly personal wellness reimbursement
- Health, Dental, Vision insurance

## Similar roles

- [Software Engineer, Robotics Data](https://hotfix.jobs/jobs/c68e72f3-b1c5-46e6-8bb6-878c4711ab8f) - Mercor - San Francisco, CA - $130k – $500k/yr
- [Data Analytics](https://hotfix.jobs/jobs/b8dd4634-fd75-4c54-8f56-f1c6d2ca6b99) - OnePay - Remote - $130k – $170k/yr
- [Data Engineer](https://hotfix.jobs/jobs/08aa197f-318d-4d15-8927-be61227caa00) - Mercor - San Francisco, CA - $130k – $500k/yr
- [Software Engineer, Research - Human Data](https://hotfix.jobs/jobs/397c3a4f-9554-4921-be00-5ab65922c1ee) - OpenAI - San Francisco, CA - $131k – $385k/yr
- [Data Center Compute](https://hotfix.jobs/jobs/b1379083-3dc5-442a-8d2d-495fd9345dca) - OpenAI - Remote - $125k – $400k/yr

**Apply:** https://hotfix.jobs/jobs/263e7264-3e8d-4036-b737-5569345eabd1
**Canonical:** https://hotfix.jobs/jobs/263e7264-3e8d-4036-b737-5569345eabd1