# Research Engineer – Evals

**Company:** [Firecrawl](https://hotfix.jobs/companies/firecrawl)
**Location:** Remote
**Role:** ML Engineering
**Salary:** $210k – $275k/yr
**Experience:** 4+ years
**Skills:** Evaluation Systems, Metrics Design, Data Pipelines, Dataset Curation, Ml Engineering, Research Engineering, Llm Evaluation, Quality Measurement
**Posted:** 2026-07-12

> Build and own evaluation systems, metrics, pipelines, and datasets to rigorously measure the quality of Firecrawl's LLM-ready web data outputs at massive scale, driving model and product improvements.

## Job Description

## What You'll Do
- Design the metrics that define what "good output" actually means across millions of sites, formats, and edge cases
- Build the pipelines and harnesses that measure quality rigorously and at scale
- Generate and curate the datasets that make evaluation trustworthy
- Own the feedback loop from output quality back to model and product decisions
- Turn "did that work?" into an answer the whole team can act on

## What We're Looking For
- Engineering depth to build real evaluation systems, not just run existing ones
- Care deeply about what "good" means and how to measure it rigorously
- Comfortable owning ambiguous problems where the metric itself has to be invented
- Move fast and close the loop - ship, measure, and iterate

## Requirements
- 4+ years in ML, research engineering, or data-heavy backend, with real evaluation work
- Must already be authorized to work in the US or our eligible remote-hire regions

## Nice-to-Haves
- Experience designing metrics, building evaluation pipelines, generating datasets for LLM/web data quality
- Background in measuring complex AI system outputs at scale

## Compensation & Benefits
**Salary:** $210,000–$275,000/year (U.S.-based in San Francisco, CA; adjusted for other locations based on cost of living)  
**Equity:** Up to 0.05%  
**Benefits (US-based):** Full medical, dental, vision (100% for employees); life & disability insurance; 401(k); generous PTO (15+ days); 12 weeks parental leave; wellness stipend; learning stipend; pet insurance.  
**SF-specific:** HQ perks, e-bike loaner.  
**Other:** Sabbatical after 4 years; team offsites.

## Similar roles

- [AI Engineer - NYC](https://hotfix.jobs/jobs/cd73c639-a9f5-442c-b769-6ce6cce3da86) - Sequence - New York, NY - $210k – $230k/yr
- [AI CAD Engineer](https://hotfix.jobs/jobs/3ddd32ba-9493-429f-824c-a4ac04559af2) - Atomicmachines - Emeryville, CA - $210k – $260k/yr
- [Machine Learning Engineer](https://hotfix.jobs/jobs/29d99cab-8993-4b5f-a132-832634513ee3) - Layer Health - Boston, MA - $210k – $250k/yr
- [Applied AI Engineer - Bay Area](https://hotfix.jobs/jobs/e5aabbc0-b3cf-415e-b0ff-f96dbe33e938) - Redis - San Francisco, CA - $211k – $264k/yr
- [Machine Learning Engineer - Embedded Insights](https://hotfix.jobs/jobs/aff6646e-3664-4f34-a9db-69fd66c3eca7) - Plaid - San Francisco, CA - $212k – $272k/yr

**Apply:** https://hotfix.jobs/jobs/5d35303f-5667-41b7-a03c-dc6f8f99313e
**Canonical:** https://hotfix.jobs/jobs/5d35303f-5667-41b7-a03c-dc6f8f99313e