# AI Engineer, Model Quality and Performance

**Company:** [Cerebras Systems](https://hotfix.jobs/companies/cerebras-systems)
**Location:** Sunnyvale, CA
**Role:** ML Engineering
**Skills:** AI Agents, Claude, Docker, Git, CI/CD, Model Evaluation, Performance Benchmarking, Python, Math/Stats, Lm-Eval-Harness
**Posted:** 2026-05-15

> Own model quality and performance for Cerebras inference by building AI agent-driven eval suites, automating benchmarking, and creating customer-specific tooling. Requires strong AI agent experience and tooling intuition.

## Job Description

## What You'll Do
- Design eval suites with AI agents in the loop. For every model release, curate a thoughtful mix of advanced, basic, long-context, and customer-use-case-specific evals. Use Claude to generate, validate, and prune candidate test cases at speed.
- Build custom evals for target customers by orchestrating AI agents to mine trajectories from their workloads and synthesize representative eval sets.
- Automate eval execution end-to-end with AI-driven pipelines on top of standard tooling (Docker, Git, CI). The goal is a system that runs itself between releases, not a script you re-run by hand.
- Build automations to forecast and benchmark model performance on Cerebras for our top customers, including modeling how fast customer-specific workloads will run in production.
- Build product-quality tooling that synthesizes quality + performance data into a single, easy-to-use view.

## Skills & Qualifications
- Experience building AI agents. You ship real systems with Claude (or equivalent) as a force multiplier. You've built things that would have been infeasible solo without AI agents in the loop.
- Strong math/stats background.
- Comfort with Docker, Git, and the standard automation stack.
- A taste for tooling design. You've shipped something that a non-engineer used without complaining. Bonus if AI helped you ship it.

## Assets
- Performance-tuning experience on custom silicon, GPUs, or FPGAs.
- Experience designing evals for agentic / coding / long-context / multimodal use cases.
- Familiarity with open-source eval frameworks (EvalScope, lm-eval-harness, etc.).

## Similar roles

- [CoDesign & NextGen Performance Engineer](https://hotfix.jobs/jobs/2cb728b7-a9ea-44b0-b16a-47364021f7be) - Cerebras Systems - Sunnyvale, CA
- [Research Engineer, Privacy](https://hotfix.jobs/jobs/3220c5d0-0e5f-4869-b133-1ed126b4208a) - OpenAI - San Francisco, CA - $380k – $445k/yr
- [Research Engineer](https://hotfix.jobs/jobs/b4d18018-e64d-4828-a8b4-303225328e17) - Console - San Francisco, CA - $200k – $350k/yr
- [Software Engineer, AI Platform](https://hotfix.jobs/jobs/7ddad000-3e1c-47cb-a45b-31d8e72abc4f) - Notion - San Francisco, CA - $180k – $201k/yr
- [Machine Learning Engineer](https://hotfix.jobs/jobs/97713038-58cb-46c5-a9c7-0e1554e1ba4f) - Liftoff - Remote - $215k – $275k/yr

**Apply:** https://hotfix.jobs/jobs/85c2e41f-f9fb-4638-95f6-5f0f12be7add
**Canonical:** https://hotfix.jobs/jobs/85c2e41f-f9fb-4638-95f6-5f0f12be7add