Skip to content
DrataDrataSan Francisco, CA

Manager, AI Engineering - Analytics

Player-coach role leading a small AI engineering team building natural language analytics and reporting features. Hands-on coding plus management responsibilities for agents, evals, and data systems.

Salary not listed
Hybrid6+ YOEML Engineering

About the role

What you'll do

Build Alongside the Team

  • Stay deeply hands-on by writing code, designing systems, and reviewing PRs
  • Own critical paths and pair with engineers on the hardest parts of the product
  • Keep close to the codebase and the customer experience even as the team grows
  • Set the bar for engineering quality through your own work

Lead a Small Team

  • Lead a small, focused team of engineers and grow it thoughtfully over time
  • Set clear goals, run good 1:1s, and create an environment where engineers do their best work
  • Give direct, useful feedback and help engineers grow in their careers
  • Invest in the basics of management: hiring, performance, career growth, and team health
  • Partner with leadership to grow into the formal management craft

Own the AI and Data Direction

  • Set the technical direction for AI-driven analytics and the data foundation underneath it
  • Make pragmatic decisions across the stack, from data modeling to agent design
  • Define multi-tenant data access patterns that safely serve customer-scoped data at scale
  • Make sound build, buy, and adopt decisions for the team's tooling
  • Stay current on developments in applied AI and bring relevant ideas back to the team

Build Natural Language Data Experiences

  • Help shape and build features that let users ask questions of their data in natural language
  • Ground AI responses in real data, handle ambiguity, and surface uncertainty appropriately
  • Keep AI-driven experiences fast, accurate, and trustworthy
  • Iterate quickly with design partners to find what works in production

Make Evals a First-Class Practice

  • Build the evals, telemetry, and offline/online test loops the team relies on
  • Establish eval-driven development as the default workflow
  • Define what "good" means for each AI feature and measure it rigorously
  • Use eval results to guide model, prompt, and architecture decisions

Ship and Learn

  • Drive end-to-end delivery from spec to GA
  • Partner with Product on scope, sequencing, and tradeoffs
  • Ship iteratively to design partners, instrument adoption, and learn from real usage
  • Establish the metrics that prove the experience is delivering value

What you'll bring

AI Engineering

  • Real AI engineering background with at least one agent or LLM-powered system shipped to production end-to-end
  • Working knowledge of prompts, tool use, retrieval, and structured outputs
  • Understanding of latency, cost, and quality tradeoffs in LLM-based systems
  • Familiarity with the failure modes of AI features in the real world

Evals

  • Hands-on experience designing and building evals for AI systems
  • Comfort with offline benchmarks, regression testing for non-deterministic systems, and online feedback loops
  • Ability to articulate how to evaluate an agent before, during, and after launch
  • Bias toward measurable quality over vibes

Data Fundamentals

  • Strong SQL skills and comfort with modern data warehouses
  • Experience with data modeling and the plumbing that powers analytics
  • Ability to reason about query performance, data contracts, and multi-tenant access patterns
  • Comfort working close to the data, not just on top of it

Hands-On and Pragmatic

  • Happy writing code and intend to keep doing it
  • Pragmatic about technology choices and careful about complexity
  • Bias toward shipping and learning over over-engineering
  • Comfortable working across the full stack on a small team

Ready to Lead

  • Track record of leading projects, mentoring engineers, and driving technical direction
  • Strong written and verbal communication
  • Direct, kind feedback style and a desire to invest in growing a team
  • Clear pull toward leadership, even without prior formal management experience

Requirements

  • 6+ years of software engineering experience, with at least 2 focused on AI/ML or applied AI work (agents, LLMs, evals, or similar)
  • At least one agent or LLM-powered system deployed to production that you owned end-to-end
  • Hands-on experience building and using evals to measure and improve AI quality
  • Solid data engineering or analytics engineering experience, including SQL, modeling, and modern data warehouses
  • Track record of shipping production software on small teams and operating across the full stack
  • Experience as a tech lead, project lead, or strong mentor, with a desire to grow into formal management
  • Strong written and verbal communication
  • Bachelor's degree in Computer Science, Engineering, or related field, or equivalent experience

Bonus Qualifications

  • Prior experience working on a customer-facing data product, embedded analytics, BI tooling, or a natural language interface over structured data (text-to-SQL, conversational analytics, or similar)
  • Experience with semantic modeling layers or modern BI infrastructure
  • Experience integrating AI agents with structured data sources
  • Background in compliance, security, GRC, or other regulated SaaS verticals
  • Prior tech lead or team lead experience
  • Previous experience at high-growth SaaS companies

Skills

PythonSQLLLMsAI AgentsEvalsPrompt EngineeringRAGData ModelingData WarehousesMulti-Tenant Systems

Similar roles

ML Engineering jobs
Scale AI

Senior Machine Learning Engineer, Agent Oversight

Scale AISan Francisco, CA +1

Senior ML Engineer building observability, evaluation frameworks, and improvement loops for production agentic AI systems. Requires 5+ years production ML/LLM experience, strong grounding in agent design or evaluation, and hands-on work taking systems from prototype to scale.

216k – 270k/yr
On-site5+ YOEML Engineering
Scale AI

Senior Software Engineer, Agent Oversight

Scale AISan Francisco, CA +1

Build platform infrastructure for observing, evaluating, and improving production AI agents at scale. Requires 4+ years software engineering experience with ML/LLM systems, strong backend/distributed systems skills, and collaboration with ML engineers.

216k – 270k/yr
On-site5+ YOEML Engineering
Ambient.ai

Senior Software Engineer, AI Infrastructure

Ambient.aiRedwood City, CA

Build and optimize scalable AI infrastructure for real-time inference, evaluation, and continuous improvement of LLMs, LVMs, computer vision, and multimodal models on large-scale video data. Requires 4+ years production ML systems experience, strong Python skills, and expertise in inference optimization and model serving.

168k – 205k/yr
Hybrid4+ YOEML Engineering
SpotOn

Senior Software Engineer

SpotOnChicago, IL +3

Senior AI Engineer building agentic workflows and orchestration layers to automate manual business processes for operations, sales, and customer support teams. Requires 7+ years software engineering experience, production LLM/agent expertise, and strong Python/TypeScript skills.

160k – 190k/yr
Hybrid7+ YOEML Engineering
Snowflake

Senior AI Engineer, Enterprise

SnowflakeMenlo Park, CA

Senior AI Engineer building scalable backend services, distributed systems, and reusable AI platform components (agent frameworks, evaluation pipelines, SDKs) that power Snowflake's internal agentic applications for GTM teams. Requires 5+ years software engineering experience, strong Python/FastAPI skills, and production LLM experience.

156k – 224k/yr
On-site5+ YOEML Engineering