Software Engineer, AI Platform
Build core AI platform infrastructure at Harvey including model routing, context management, evaluation frameworks, and shared abstractions for agentic AI products. Requires 5+ years backend experience with 1+ year in AI/ML, production multi-model systems, and strong platform-building skills.
About the job
What You’ll Do
- Design and build abstractions and platform-level systems that improve all of Harvey’s agentic products.
- Own infrastructure for model integration, routing, and evaluation that helps Harvey choose and deploy the right foundation model for any given context.
- Build evaluation frameworks and tooling that let every team across Harvey iterate on AI quality effectively.
- Partner closely with product engineering teams, PMs, and design to launch cutting-edge AI products.
- Evaluate, prototype, and integrate the latest advancements in AI and agentic systems as they emerge.
What You Have
- 5+ years of experience building backend systems, with at least 1+ year focused on AI/ML engineering. Staff candidates will typically have 8+ years and a track record of technical leadership across teams.
- Experience building and shipping multi-model or multi-provider AI systems in production.
- Familiarity with context management, session state, or memory systems in AI or distributed systems.
- A track record of building internal platforms, SDKs, or shared infrastructure that other engineering teams actually adopted.
- Strong judgment about abstractions and opinionated about good design but pragmatic about shipping incrementally.
- Excitement about agentic AI and the infrastructure challenges of making autonomous systems reliable.
- A bias toward full ownership: you navigate ambiguity well and don’t wait for a roadmap to start solving problems.
Bonus
- Experience building evaluation frameworks.
- Working with agent/function-calling architectures.
- Familiarity with legal or other high-stakes professional services domains.
- Time at early-stage or hyper-growth startups where the underlying technology changes regularly.
Compensation
$220,000 - $300,000
Skills
Backend Systems, Ai/Ml Engineering, Multi-Model Ai Systems, Model Routing, Context Management, Evaluation Frameworks, SDKs, Agentic AI, Function Calling, Distributed Systems
Similar jobs
ML Engineering jobs
Earnin
Mountain View, CA
Build production-grade AI agents, evaluation infrastructure, and developer tooling that make AI-assisted engineering faster, safer, and reusable across teams. The role requires software engineering experience, platform or internal developer-product experience, and hands-on expertise with LLM integration and orchestration.
Baseten
San Francisco, CA
Build and ship AI-powered agents, automations, dashboards, and integrations that improve GPU capacity operations across Compute and C3. The role requires 3+ years of AI automation or technical operations experience, production Vercel expertise, agent-tool fluency, and strong API integration skills.
Mirage
New York, NY
Build and deploy agentic systems that power AI-driven creative video workflows. The role requires 5+ years of experience, production ML or agentic pipeline development, context engineering, and expertise in evaluation and agent infrastructure.
Mirage
New York, NY
Build and advance agentic machine-learning systems for multimodal creative tasks, with a focus on video understanding, reasoning, control, and tool use. The role requires strong production ML or agent-pipeline experience and deep knowledge of modern LLM techniques.
Taste Labs
San Francisco, CA
Build evaluation methods, RL environments, agent tooling, and scalable infrastructure that make subjective qualities such as design and taste measurable for frontier AI models. The role requires experience with evaluations, RL environments, ML or post-training, plus strong backend engineering skills.