Skip to content
HarveyHarveySan Francisco, CA

Software Engineer, Agents

Builds AI agents for legal workflows, optimizing performance via prompt engineering, model selection, tools, and evaluations. Partners with PMs and customers to ship low-latency, high-quality agent systems using Python and LLM APIs. Requires 2-5 years experience.

161k – 242k
Hybrid2+ YOEML Engineering

About the role

What You'll Do

  • Partner with customers and PMs to understand legal workflows, design practical evaluations that capture what “excellent” means, and ship agents that get the job done.
  • Optimize agent performance through prompt engineering, model selection, tool design, skill writing, context window management, and eval harness development.
  • Work with our model infra team to design and implement infrastructure for low-latency agent execution, including caching strategies, parallel tool calls, or subagent patterns.
  • Improve our observability and instrumentation to profile agent behavior, identify bottlenecks, and drive optimization decisions.
  • Stay current on new developments in agentic systems and bring those learnings back to the products we build.

What You Have

  • Passion for building effective domain-specific agents.
  • Iterative mindset: you develop proof of concepts, make decisions quickly, and ship v0s.
  • Comfortable with when and how to use evaluations to drive quality.
  • Humble and adaptable about code and frameworks. We expect you to drive adoption of new best practices as they develop.
  • 2-5 years (post-BS/MS) of software engineering experience. We are hiring for this role across Mid, Senior, and Staff levels.
  • Proficiency in Python and experience working with LLM APIs and agent frameworks.
  • Experience with shipping user-facing products, either on the backend or full-stack.

Compensation Range

$161,300 - $241,900 USD

Skills

PythonLLM APIsAgent FrameworksPrompt EngineeringModel SelectionTool DesignContext Window ManagementEval HarnessObservabilityInstrumentation

Similar roles

ML Engineering jobs
Nuro

Software Engineer, ML Infrastructure, Optimization

NuroMountain View, CA

Build and optimize ML infrastructure for autonomous vehicles, focusing on model optimization, compilers, and deployment across the autonomy stack. Requires 2+ years in ML optimization and strong Python/C++/CUDA skills.

160k – 241k
On-site2+ YOEML Engineering
Baseten

Software Engineer, Model Performance Tooling

BasetenSan Francisco, CA

Builds performance benchmarking, diagnostic, and optimization tools for LLM inference on GPU clusters. Early-career role requiring Python proficiency, systems curiosity, and interest in AI hardware—no prior experience needed.

160k – 200k
On-siteEntry levelML Engineering
Pinterest

Machine Learning Engineer, Monetization Engineering

PinterestSan Francisco, CA +2

Develops and evolves machine learning technology stack for Pinterest's Ads monetization, building personalized recommendation systems using deep learning and LLMs. Requires 2+ years ML experience, CS degree, and expertise in large-scale systems.

163k – 286k
Hybrid2+ YOEML Engineering
Abridge

Software Engineer

AbridgeSan Francisco, CA +1

Early-career Software Engineer building and iterating on agentic LLM systems, RAG pipelines, evaluation frameworks, and backend/frontend components for AI-powered clinical documentation at Abridge. Requires CS degree or equivalent plus hands-on GenAI experience from projects, internships or coursework.

157k – 184k
On-siteEntry levelML Engineering
Airbnb

Machine Learning Engineer, Relevance and Personalization

AirbnbUnited States

Build and productionize cutting-edge ML models and ranking algorithms for Airbnb's search, recommendation, and personalization systems at scale. Requires PhD (new grad) or 2+ years applied ML experience with strong programming and ML systems skills.

166k – 195k
Remote2+ YOEML Engineering