AI Product Engineer
Build agentic capabilities on a petabyte-scale observability platform. Own the full agent stack including context engineering, tool design, evals, and production reliability for incident investigation.
Software Engineer, Applied AI
Builds and operates scalable data pipelines and systems for post-training workflows, model evaluations, and synthetic data generation. Partners with frontier AI labs and customers, requiring strong backend skills in Python/Go/Rust and ML evaluation expertise.
130k – 500kSan Francisco, CAML EngineeringOnsite
About the role
What You’ll Do
- Partner closely with frontier AI labs to understand their data, post-training, and evaluation needs
- Build and operate scalable data pipelines for post-training workflows and model evaluations
- Design and build scalable systems for synthetic data generation and data quality, and work directly with customers to understand requirements and develop technical solutions
- Prototype new data types, benchmarks, and evaluation frameworks
- Lead technical discussions with customers
What We’re Looking For
Strong backend engineering fundamentals in a modern language (Python, Go, Rust, etc.)
- Experience with model training and inference
- Strong grounding in statistical analysis and experimental design for measuring model performance and improvements
- Familiarity with evaluation methods for large language models
- Comfort working through ambiguity and shipping iteratively
You’re likely someone who:
- Enjoys ownership and customer-facing problem solving
- Thinks entrepreneurially and moves quickly
- Balances speed with engineering rigor
- Communicates clearly with technical and non-technical users
Benefits
- Bi-annual performance bonus structure
- Generous equity grant vested over 4 years
- Up to $15k Relocation bonus
- $10K housing bonus (if you live within 0.5 miles of our office)
- $1.5K monthly stipend for meals
- Free Equinox membership
- $200 monthly laundry reimbursement
- $200 monthly personal wellness reimbursement
- Health, Dental, Vision insurance
Skills
PythonGoRustModel TrainingModel InferenceStatistical AnalysisExperimental DesignLLMsData PipelinesSynthetic Data Generation
Similar roles
ML Engineering jobsSoftware Engineer, AI Data & Evaluation
As a Senior Software Engineer, AI Data & Evaluation, you will build data infrastructure and evaluation systems for frontier AI models. This role involves designing evaluation methodologies, building synthetic data generation systems, and architecting operational automation.
Machine Learning Engineer, Marketplace
Build ML models and decision systems for search, ranking, candidate-job matching, and marketplace optimization at a fast-growing AI talent platform.
Software Engineer, Applied AI
Builds and deploys custom integrations, APIs, and scalable solutions for enterprise clients using Mercor's AI platform. Requires strong engineering skills in modern languages and cloud environments, with customer-facing experience.
Research Engineer – Benchmarking, Evals & Failure Analysis
Designs and maintains benchmarks, evaluation systems, and failure analysis workflows for frontier LLMs, focusing on tool use, agentic behavior, and reasoning. Builds rubrics, scorers, and dashboards while collaborating with AI research teams in a fast-paced environment.