# Software Engineer, AI Platform

**Company:** [Harvey](https://hotfix.jobs/companies/harvey)
**Location:** New York, NY, San Francisco, CA
**Role:** Backend Engineering
**Salary:** $220k – $300k/yr
**Experience:** 5+ years
**Skills:** AI/ML, Backend Systems, Model Routing, Context Management, Agent Infrastructure, Evaluation Frameworks, SDKs, Distributed Systems, Foundation Models, Function-Calling
**Posted:** 2026-03-12

> Builds AI platform foundation including model routing, agent infrastructure, context management, and evaluation systems for legal AI products. Requires 5+ years backend experience with 1+ year AI/ML, production multi-model systems, and shared platform expertise.

## Job Description

## Representative Projects
- **Context Engineering & Agent Infrastructure**: Build platform-level systems for context management, session state, and memory.
- **Model Integration & Routing**: Own infrastructure for onboarding foundation models and routing to the right one.
- **Evaluation Infrastructure**: Build shared eval tooling and frameworks.
- **Shared Abstractions**: Create SDKs, platform primitives, and developer tooling.

## What You’ll Do
- Design and build abstractions and platform-level systems that improve all agentic products.
- Own infrastructure for model integration, routing, and evaluation.
- Build evaluation frameworks and tooling for AI quality iteration.
- Partner with product engineering teams, PMs, and design to launch AI products.
- Evaluate, prototype, and integrate latest AI and agentic systems advancements.

## What You Have
- 5+ years building backend systems, with 1+ year in AI/ML engineering (Staff: 8+ years, technical leadership).
- Experience shipping multi-model or multi-provider AI systems in production.
- Familiarity with context management, session state, or memory in AI/distributed systems.
- Track record building internal platforms, SDKs, or shared infrastructure adopted by teams.
- Strong judgment on abstractions; pragmatic about shipping.
- Excitement for agentic AI infrastructure challenges.
- Bias toward full ownership in ambiguity.

**Bonus**: Evaluation frameworks, agent/function-calling, legal domain, early-stage/hyper-growth experience.

## Compensation
$220,000 - $300,000

## Similar roles

- [Software Engineer, Foundations](https://hotfix.jobs/jobs/86894974-73e3-4822-897b-87f5aae9fc5f) - Hex - Remote - $215k – $270k/yr
- [Software Engineer, Codex -Enterprise Controls](https://hotfix.jobs/jobs/9fc148ac-afcb-4558-a41e-a772e8869a71) - OpenAI - San Francisco, CA - $230k – $385k/yr
- [Software Engineer, Private Computing](https://hotfix.jobs/jobs/61850697-3157-406e-8968-ea0a1b8753cd) - OpenAI - San Francisco, CA - $230k – $385k/yr
- [Software Engineer, Enterprise AI Platform](https://hotfix.jobs/jobs/487f3501-620e-4e9d-abf9-8c3e9c0d2034) - OpenAI - San Francisco, CA - $230k – $385k/yr
- [Software Engineer, APIs & Context Platform](https://hotfix.jobs/jobs/8072b048-afc5-451e-ace0-4bdec5dd1411) - Glean - Mountain View, CA - $210k – $260k/yr

**Apply:** https://hotfix.jobs/jobs/19abe607-534e-4538-8af6-082d2d43eae2
**Canonical:** https://hotfix.jobs/jobs/19abe607-534e-4538-8af6-082d2d43eae2