Manager, AI Engineering - Analytics
San Francisco, CAML EngineeringHybrid6+ YOE
Summary
Player-coach role leading a small AI engineering team building natural language analytics and reporting features. Hands-on coding plus management responsibilities for agents, evals, and data systems.
About the role
What you'll do
Build Alongside the Team
- Stay deeply hands-on by writing code, designing systems, and reviewing PRs
- Own critical paths and pair with engineers on the hardest parts of the product
- Keep close to the codebase and the customer experience even as the team grows
- Set the bar for engineering quality through your own work
Lead a Small Team
- Lead a small, focused team of engineers and grow it thoughtfully over time
- Set clear goals, run good 1:1s, and create an environment where engineers do their best work
- Give direct, useful feedback and help engineers grow in their careers
- Invest in the basics of management: hiring, performance, career growth, and team health
- Partner with leadership to grow into the formal management craft
Own the AI and Data Direction
- Set the technical direction for AI-driven analytics and the data foundation underneath it
- Make pragmatic decisions across the stack, from data modeling to agent design
- Define multi-tenant data access patterns that safely serve customer-scoped data at scale
- Make sound build, buy, and adopt decisions for the team's tooling
- Stay current on developments in applied AI and bring relevant ideas back to the team
Build Natural Language Data Experiences
- Help shape and build features that let users ask questions of their data in natural language
- Ground AI responses in real data, handle ambiguity, and surface uncertainty appropriately
- Keep AI-driven experiences fast, accurate, and trustworthy
- Iterate quickly with design partners to find what works in production
Make Evals a First-Class Practice
- Build the evals, telemetry, and offline/online test loops the team relies on
- Establish eval-driven development as the default workflow
- Define what "good" means for each AI feature and measure it rigorously
- Use eval results to guide model, prompt, and architecture decisions
Ship and Learn
- Drive end-to-end delivery from spec to GA
- Partner with Product on scope, sequencing, and tradeoffs
- Ship iteratively to design partners, instrument adoption, and learn from real usage
- Establish the metrics that prove the experience is delivering value
What you'll bring
AI Engineering
- Real AI engineering background with at least one agent or LLM-powered system shipped to production end-to-end
- Working knowledge of prompts, tool use, retrieval, and structured outputs
- Understanding of latency, cost, and quality tradeoffs in LLM-based systems
- Familiarity with the failure modes of AI features in the real world
Evals
- Hands-on experience designing and building evals for AI systems
- Comfort with offline benchmarks, regression testing for non-deterministic systems, and online feedback loops
- Ability to articulate how to evaluate an agent before, during, and after launch
- Bias toward measurable quality over vibes
Data Fundamentals
- Strong SQL skills and comfort with modern data warehouses
- Experience with data modeling and the plumbing that powers analytics
- Ability to reason about query performance, data contracts, and multi-tenant access patterns
- Comfort working close to the data, not just on top of it
Hands-On and Pragmatic
- Happy writing code and intend to keep doing it
- Pragmatic about technology choices and careful about complexity
- Bias toward shipping and learning over over-engineering
- Comfortable working across the full stack on a small team
Ready to Lead
- Track record of leading projects, mentoring engineers, and driving technical direction
- Strong written and verbal communication
- Direct, kind feedback style and a desire to invest in growing a team
- Clear pull toward leadership, even without prior formal management experience
Requirements
- 6+ years of software engineering experience, with at least 2 focused on AI/ML or applied AI work (agents, LLMs, evals, or similar)
- At least one agent or LLM-powered system deployed to production that you owned end-to-end
- Hands-on experience building and using evals to measure and improve AI quality
- Solid data engineering or analytics engineering experience, including SQL, modeling, and modern data warehouses
- Track record of shipping production software on small teams and operating across the full stack
- Experience as a tech lead, project lead, or strong mentor, with a desire to grow into formal management
- Strong written and verbal communication
- Bachelor's degree in Computer Science, Engineering, or related field, or equivalent experience
Bonus Qualifications
- Prior experience working on a customer-facing data product, embedded analytics, BI tooling, or a natural language interface over structured data (text-to-SQL, conversational analytics, or similar)
- Experience with semantic modeling layers or modern BI infrastructure
- Experience integrating AI agents with structured data sources
- Background in compliance, security, GRC, or other regulated SaaS verticals
- Prior tech lead or team lead experience
- Previous experience at high-growth SaaS companies
Skills
PythonSQLLLMsAI AgentsEvalsPrompt EngineeringRAGData ModelingData WarehousesMulti-tenant Systems