Agent Reliability Engineer, GTM

Responsibilities

• Monitor production health across agent graphs, identifying errors, slow or expensive runs, and silent failures.
• Triage issues from Slack, tickets, and representative reports; resolve smaller issues and route others to the appropriate owner.
• Run weekly evaluation suites, investigate failures, and convert production bugs into regression tests.
• Track cost and latency by model, graph, use case, and role; recommend improvements to model selection, reasoning effort, and caching.
• Track usage and adoption by representative and feature, and own weekly health reporting.
• Build business metrics covering reply rates, meetings booked, hours reclaimed, and ROI.
• Build monitoring and alerting on LangSmith and document how the company operates its own agent for customer-facing reference material.

Requirements

• Strong production Python and SQL skills, with comfort working in traces, logs, and warehouse tables.
• Experience operating LLM applications, including tracing, evaluations, prompts, and caching.
• SRE or production-operations instincts, including percentiles, SLOs, and distinguishing meaningful patterns from noise.
• Healthy skepticism about metrics and careful validation of what metrics measure.
• Clear writing skills and interest in publishing technical learnings.
• High agency and initiative.

Nice to Have

• LangGraph or LangSmith experience.
• Experience building an evaluation suite from scratch.
• BigQuery or dbt experience.
• Developer-relations-adjacent writing.
• Empathy for sales and go-to-market users.

Compensation

• Base salary: $150,000–$190,000.
• Compensation may also include variable compensation, equity, benefits, and perks.
• Benefits include medical, dental, and vision coverage, flexible vacation, a 401(k) plan, and meals on in-office days in the United States.