Skip to content
HarveyHarveySan Francisco, CA

Mid/Senior/Staff Software Engineer, Agents

Build AI agent systems for legal workflows, optimizing performance via prompt engineering, model selection, tools, and evals. Requires 3+ years experience, Python proficiency, and LLM/agent framework expertise for mid/senior/staff levels.

165k – 312k
On-siteML Engineering

About the role

Representative Projects

  • Save weeks of effort on high pressure M&A deals by automating information requests and diligence checks across hundreds of thousands of files with retrieval and file editing agents.
  • Dramatically speed up deal turnaround speed by improving the latency and quality of agents on applying a standard legal "playbook" to contracts.
  • Ensure that our clients win their complex, data-hungry litigation cases by optimizing our multi-source retrieval agents.
  • Enable board-ready PowerPoint generation by tuning the harness and libraries for coding agents.

What You'll Do

  • Partner with customers and PMs to understand legal workflows, design practical evaluations that capture what “excellent” means, and ship agents that get the job done.
  • Optimize agent performance through prompt engineering, model selection, tool design, skill writing, context window management, and eval harness development.
  • Work with our model infra team to design and implement infrastructure for low-latency agent execution, including caching strategies, parallel tool calls, or subagent patterns.
  • Improve our observability and instrumentation to profile agent behavior, identify bottlenecks, and drive optimization decisions.
  • Stay current on new developments in agentic systems and bring those learnings back to the products we build.

What You Have

  • Passion for building effective domain-specific agents.
  • Iterative mindset: you develop proof of concepts, make decisions quickly, and ship v0s.
  • Comfortable with when and how to use evaluations to drive quality.
  • Humble and adaptable about code and frameworks. We expect you to drive adoption of new best practices as they develop.
  • 3+ years (post-BS/MS) of software engineering experience.
  • We are hiring for this role across Mid, Senior, and Staff levels.
  • Proficiency in Python and experience working with LLM APIs and agent frameworks.
  • Experience with shipping user-facing products, either on the backend or full-stack.

Compensation Range

$165,000 - $312,000 USD

Skills

PythonLLM APIsAgent FrameworksPrompt EngineeringEvaluationsObservabilityModel SelectionContext Window ManagementTool DesignCaching Strategies

Similar roles

ML Engineering jobs
Mozilla

Senior Staff AI & Agentic Systems Engineer

MozillaUnited States

Lead architectural design and implementation of production-grade multi-agent frameworks, LLM orchestration, and autonomous AI agent systems from the ground up at Mozilla's New Products incubator. Requires 7+ years software engineering experience including 2+ years building agentic systems, startup mentality, and deep proficiency with AI tooling.

166k – 260k
Remote7+ YOEML Engineering
Gusto

Retirement AI Staff Engineer

GustoDenver, CO +2

Staff Engineer building and shipping LLM-powered retirement features end-to-end. Owns architecture decisions, integrates models, and collaborates closely with product and design.

164k – 247k
Hybrid7+ YOEML Engineering
Databricks

Staff Software Engineer, AI Search

DatabricksMountain View, CA

As a Staff Engineer for Search, you will build and scale Databricks' next-generation Search product, driving the design and evolution of a highly-performant, cost-efficient, and developer-friendly Search stack. You will also define the long-term vision, mentor senior engineers, and lead strategic efforts.

166k – 225k
On-site10+ YOEML Engineering
Square

Staff Machine Learning Engineer, Modeling, Support

SquareSeattle, WA

Lead end-to-end delivery of ML initiatives for customer support, focusing on conversational AI and generative AI. Partner with cross-functional teams to define ML roadmaps and drive R&D efforts for next-generation chatbot architectures.

161k – 284k
On-site10+ YOEML Engineering
Clear Street

Senior / Staff Software Engineer

Clear StreetUnited States

Build high-performance Rust backend and core AI platform powering an AI-native trading copilot with streaming, low-latency tools, safe trading actions, and robust APIs. Requires 8+ years experience in systems programming, production services, and architecture.

170k – 240k
Remote8+ YOEML Engineering