Skip to content
HarveyHarveySan Francisco, CA

Staff Software Engineer, AI Platform

Build core AI platform infrastructure at Harvey including model routing, context engineering, agent infrastructure, and shared evaluation frameworks that power all agentic legal AI products. Requires 8+ years backend experience with 1+ year AI/ML focus and track record of technical leadership.

231k – 340k/yr
Hybrid8+ YOEML Engineering

About the role

What You’ll Do

  • Design and build abstractions and platform-level systems that improve all of Harvey’s agentic products.
  • Own infrastructure for model integration, routing, and evaluation that helps Harvey choose and deploy the right foundation model for any given context.
  • Build evaluation frameworks and tooling that let every team across Harvey iterate on AI quality effectively.
  • Partner closely with product engineering teams, PMs, and design to launch cutting-edge AI products.
  • Evaluate, prototype, and integrate the latest advancements in AI and agentic systems as they emerge.

What You Have

  • 8+ years of experience building backend systems, with at least 1+ year focused on AI/ML engineering and a track record of technical leadership across teams.
  • Experience building and shipping multi-model or multi-provider AI systems in production.
  • Familiarity with context management, session state, or memory systems in AI or distributed systems.
  • A track record of building internal platforms, SDKs, or shared infrastructure that other engineering teams actually adopted.
  • Strong judgment about abstractions and opinionated about good design but pragmatic about shipping incrementally.
  • Excitement about agentic AI and the infrastructure challenges of making autonomous systems reliable.
  • A bias toward full ownership: navigate ambiguity well and don’t wait for a roadmap to start solving problems.

Bonus

  • Experience building evaluation frameworks.
  • Working with agent/function-calling architectures.
  • Familiarity with legal or other high-stakes professional services domains.
  • Time at early-stage or hyper-growth startups where the underlying technology changes regularly.

Skills

ai/ml engineeringBackend Systemsmodel routingmulti-model ai systemscontext managementEvaluation FrameworksSDKsagent architecturesfunction callingDistributed Systems

Similar roles

ML Engineering jobs
Harvey

Staff Software Engineer, Agents

HarveySan Francisco, CA +1

Builds AI agent systems for legal workflows, optimizing performance via prompt engineering, model selection, tools, and evaluations. Partners with customers/PMs to ship low-latency agents using Python and LLM APIs; requires 8+ years experience.

231k – 340k/yr
Hybrid8+ YOEML Engineering
Reddit

Staff Machine Learning Engineer, Notifications Relevance

RedditUnited States

Technical leader for Reddit's Notifications Relevance ML systems, driving large-scale recommendation systems spanning retrieval, ranking, budget optimization, and LLM-powered experiences.

230k – 322k/yr
Remote8+ YOEML Engineering
Reddit

Staff Machine Learning Engineer, Ads Measurement Modeling

RedditUnited States

Leads technical strategy, architecture, and development of advanced ML models for ads identity modeling and measurement, ensuring scalability, privacy compliance, and integration across systems. Requires 7+ years software engineering with 3+ in ML at scale, expertise in ML frameworks and large-scale data processing.

230k – 322k/yr
Remote7+ YOEML Engineering
Fractional AI

Senior / Staff Software Engineer (SF/NY)

Fractional AISan Francisco, CA +1

Senior/Staff Software Engineer builds end-to-end AI products for clients, spending 75% time coding and 25% with stakeholders like CTOs. Requires 8+ years experience shipping products, applying software engineering to AI systems with high ownership.

230k – 350k/yr
Hybrid8+ YOEML Engineering
Betterment

Sr. Staff Engineer

BettermentNew York, NY

Leads AI platform strategy and infrastructure for deploying AI features across Betterment's products, partnering with product, compliance, and engineering teams. Requires deep LLM expertise, fullstack experience with React/GraphQL/server languages, and production AI operations in a regulated environment.

230k – 270k/yr
HybridML Engineering