# Staff Software Engineer, AI Platform

**Company:** [Harvey](https://hotfix.jobs/companies/harvey)
**Location:** San Francisco, CA, New York, NY
**Role:** ML Engineering
**Salary:** $231k – $340k/yr
**Experience:** 8+ years
**Skills:** ai/ml engineering, Backend Systems, model routing, multi-model ai systems, context management, Evaluation Frameworks, SDKs, agent architectures, function calling, Distributed Systems
**Posted:** 2026-04-24

> Build core AI platform infrastructure at Harvey including model routing, context engineering, agent infrastructure, and shared evaluation frameworks that power all agentic legal AI products. Requires 8+ years backend experience with 1+ year AI/ML focus and track record of technical leadership.

## Job Description

## What You’ll Do
- Design and build abstractions and platform-level systems that improve all of Harvey’s agentic products.
- Own infrastructure for model integration, routing, and evaluation that helps Harvey choose and deploy the right foundation model for any given context.
- Build evaluation frameworks and tooling that let every team across Harvey iterate on AI quality effectively.
- Partner closely with product engineering teams, PMs, and design to launch cutting-edge AI products.
- Evaluate, prototype, and integrate the latest advancements in AI and agentic systems as they emerge.

## What You Have
- 8+ years of experience building backend systems, with at least 1+ year focused on AI/ML engineering and a track record of technical leadership across teams.
- Experience building and shipping multi-model or multi-provider AI systems in production.
- Familiarity with context management, session state, or memory systems in AI or distributed systems.
- A track record of building internal platforms, SDKs, or shared infrastructure that other engineering teams actually adopted.
- Strong judgment about abstractions and opinionated about good design but pragmatic about shipping incrementally.
- Excitement about agentic AI and the infrastructure challenges of making autonomous systems reliable.
- A bias toward full ownership: navigate ambiguity well and don’t wait for a roadmap to start solving problems.

## Bonus
- Experience building evaluation frameworks.
- Working with agent/function-calling architectures.
- Familiarity with legal or other high-stakes professional services domains.
- Time at early-stage or hyper-growth startups where the underlying technology changes regularly.

## Similar roles

- [Staff Software Engineer, Agents](https://hotfix.jobs/jobs/a97dd7fb-bd69-4449-8010-971ab051bbfa) - Harvey - San Francisco, CA - $231k – $340k/yr
- [Staff Machine Learning Engineer, Notifications Relevance](https://hotfix.jobs/jobs/5df4bbf6-982b-4563-aaee-d477695b1557) - Reddit - Remote - $230k – $322k/yr
- [Staff Machine Learning Engineer, Ads Measurement Modeling](https://hotfix.jobs/jobs/66ae67c8-cd05-42e4-9ea4-9f913dc37d43) - Reddit - Remote - $230k – $322k/yr
- [Senior / Staff Software Engineer (SF/NY)](https://hotfix.jobs/jobs/21b3e63e-16d5-4495-a4eb-cb797436d773) - Fractional AI - San Francisco, CA - $230k – $350k/yr
- [Sr. Staff Engineer](https://hotfix.jobs/jobs/592ff4db-35fb-49ec-9f4c-1aff059846c7) - Betterment - New York, NY - $230k – $270k/yr

**Apply:** https://hotfix.jobs/jobs/0786beca-d2bb-4de3-8134-cd87ca1563aa
**Canonical:** https://hotfix.jobs/jobs/0786beca-d2bb-4de3-8134-cd87ca1563aa