Staff Software Engineer, AI Platform
Build core AI platform infrastructure at Harvey including model routing, context engineering, agent infrastructure, and shared evaluation frameworks that power all agentic legal AI products. Requires 8+ years backend experience with 1+ year AI/ML focus and track record of technical leadership.
About the job
What You’ll Do
- Design and build abstractions and platform-level systems that improve all of Harvey’s agentic products.
- Own infrastructure for model integration, routing, and evaluation that helps Harvey choose and deploy the right foundation model for any given context.
- Build evaluation frameworks and tooling that let every team across Harvey iterate on AI quality effectively.
- Partner closely with product engineering teams, PMs, and design to launch cutting-edge AI products.
- Evaluate, prototype, and integrate the latest advancements in AI and agentic systems as they emerge.
What You Have
- 8+ years of experience building backend systems, with at least 1+ year focused on AI/ML engineering and a track record of technical leadership across teams.
- Experience building and shipping multi-model or multi-provider AI systems in production.
- Familiarity with context management, session state, or memory systems in AI or distributed systems.
- A track record of building internal platforms, SDKs, or shared infrastructure that other engineering teams actually adopted.
- Strong judgment about abstractions and opinionated about good design but pragmatic about shipping incrementally.
- Excitement about agentic AI and the infrastructure challenges of making autonomous systems reliable.
- A bias toward full ownership: navigate ambiguity well and don’t wait for a roadmap to start solving problems.
Bonus
- Experience building evaluation frameworks.
- Working with agent/function-calling architectures.
- Familiarity with legal or other high-stakes professional services domains.
- Time at early-stage or hyper-growth startups where the underlying technology changes regularly.
Skills
Ai/Ml Engineering, Backend Systems, Model Routing, Multi-Model Ai Systems, Context Management, Evaluation Frameworks, SDKs, Agent Architectures, Function Calling, Distributed Systems
Similar jobs
ML Engineering jobs
Harvey
San Francisco, CA
Leads the design and operation of reliable, scalable model infrastructure powering AI inference across multiple providers. Requires 7+ years of distributed-systems engineering experience, strong programming skills, and expertise in production reliability and cloud infrastructure.
Shield AI
San Mateo, CA
Leads the design, implementation, integration, and field validation of tactical autonomy and multi-agent coordination capabilities for unmanned platforms. Requires 7+ years of relevant experience, production C++, technical leadership, and eligibility for a U.S. Secret clearance.
Snowflake
Bellevue, WA
Leads the roadmap and technical vision for Snowflake Feature Store, building reliable, high-performance machine learning platform capabilities and supporting technical execution across partner teams. Requires 10+ years of experience with data-serving infrastructure or ML platforms, plus Java and Python expertise.
Shield AI
Washington, DCLeads development and integration of advanced maritime autonomy for USVs, UUVs, and cooperating UAVs, including motion planning, localization, safety, and multi-agent coordination. Requires staff-level technical leadership, substantial robotics experience, C++ and Python proficiency, and eligibility for a SECRET clearance.
Ironclad
San Francisco, CA
Leads architecture and technical direction for agentic search systems combining LLMs, retrieval, and content-understanding pipelines for contract intelligence. The role requires 10+ years building production systems, deep search or LLM expertise, and strong cross-team technical leadership.