Skip to content

Manager, AI Engineering - Analytics

San Francisco, CAML EngineeringHybrid6+ YOE
Summary

Player-coach role leading a small AI engineering team building natural language analytics and reporting features. Hands-on coding plus management responsibilities for agents, evals, and data systems.

About the role

What you'll do

Build Alongside the Team

  • Stay deeply hands-on by writing code, designing systems, and reviewing PRs
  • Own critical paths and pair with engineers on the hardest parts of the product
  • Keep close to the codebase and the customer experience even as the team grows
  • Set the bar for engineering quality through your own work

Lead a Small Team

  • Lead a small, focused team of engineers and grow it thoughtfully over time
  • Set clear goals, run good 1:1s, and create an environment where engineers do their best work
  • Give direct, useful feedback and help engineers grow in their careers
  • Invest in the basics of management: hiring, performance, career growth, and team health
  • Partner with leadership to grow into the formal management craft

Own the AI and Data Direction

  • Set the technical direction for AI-driven analytics and the data foundation underneath it
  • Make pragmatic decisions across the stack, from data modeling to agent design
  • Define multi-tenant data access patterns that safely serve customer-scoped data at scale
  • Make sound build, buy, and adopt decisions for the team's tooling
  • Stay current on developments in applied AI and bring relevant ideas back to the team

Build Natural Language Data Experiences

  • Help shape and build features that let users ask questions of their data in natural language
  • Ground AI responses in real data, handle ambiguity, and surface uncertainty appropriately
  • Keep AI-driven experiences fast, accurate, and trustworthy
  • Iterate quickly with design partners to find what works in production

Make Evals a First-Class Practice

  • Build the evals, telemetry, and offline/online test loops the team relies on
  • Establish eval-driven development as the default workflow
  • Define what "good" means for each AI feature and measure it rigorously
  • Use eval results to guide model, prompt, and architecture decisions

Ship and Learn

  • Drive end-to-end delivery from spec to GA
  • Partner with Product on scope, sequencing, and tradeoffs
  • Ship iteratively to design partners, instrument adoption, and learn from real usage
  • Establish the metrics that prove the experience is delivering value

What you'll bring

AI Engineering

  • Real AI engineering background with at least one agent or LLM-powered system shipped to production end-to-end
  • Working knowledge of prompts, tool use, retrieval, and structured outputs
  • Understanding of latency, cost, and quality tradeoffs in LLM-based systems
  • Familiarity with the failure modes of AI features in the real world

Evals

  • Hands-on experience designing and building evals for AI systems
  • Comfort with offline benchmarks, regression testing for non-deterministic systems, and online feedback loops
  • Ability to articulate how to evaluate an agent before, during, and after launch
  • Bias toward measurable quality over vibes

Data Fundamentals

  • Strong SQL skills and comfort with modern data warehouses
  • Experience with data modeling and the plumbing that powers analytics
  • Ability to reason about query performance, data contracts, and multi-tenant access patterns
  • Comfort working close to the data, not just on top of it

Hands-On and Pragmatic

  • Happy writing code and intend to keep doing it
  • Pragmatic about technology choices and careful about complexity
  • Bias toward shipping and learning over over-engineering
  • Comfortable working across the full stack on a small team

Ready to Lead

  • Track record of leading projects, mentoring engineers, and driving technical direction
  • Strong written and verbal communication
  • Direct, kind feedback style and a desire to invest in growing a team
  • Clear pull toward leadership, even without prior formal management experience

Requirements

  • 6+ years of software engineering experience, with at least 2 focused on AI/ML or applied AI work (agents, LLMs, evals, or similar)
  • At least one agent or LLM-powered system deployed to production that you owned end-to-end
  • Hands-on experience building and using evals to measure and improve AI quality
  • Solid data engineering or analytics engineering experience, including SQL, modeling, and modern data warehouses
  • Track record of shipping production software on small teams and operating across the full stack
  • Experience as a tech lead, project lead, or strong mentor, with a desire to grow into formal management
  • Strong written and verbal communication
  • Bachelor's degree in Computer Science, Engineering, or related field, or equivalent experience

Bonus Qualifications

  • Prior experience working on a customer-facing data product, embedded analytics, BI tooling, or a natural language interface over structured data (text-to-SQL, conversational analytics, or similar)
  • Experience with semantic modeling layers or modern BI infrastructure
  • Experience integrating AI agents with structured data sources
  • Background in compliance, security, GRC, or other regulated SaaS verticals
  • Prior tech lead or team lead experience
  • Previous experience at high-growth SaaS companies
Skills
PythonSQLLLMsAI AgentsEvalsPrompt EngineeringRAGData ModelingData WarehousesMulti-tenant Systems