Skip to content
FetchFetchUnited States

Automation Lead, AI Operations

Leads automation and evaluation programs for AI Operations, architecting scalable workflows with LLMs, datasets, and quality systems. Requires 5+ years leading complex AI initiatives, cross-functional influence, and metrics-driven impact; familiarity with data pipelines preferred.

97k – 114k
Remote5+ YOEML Engineering

About the role

Role Responsibilities

  • Own programs: Lead complex, high-impact automation and evaluation initiatives across workflows or teams
  • Design scalable solutions: Architect end-to-end workflows integrating datasets, evaluations, automations, and HITL processes
  • Establish standards: Define best practices for datasets, evaluation methodologies, and quality measurement
  • Drive cross-functional alignment: Influence priorities and partner with Engineering, Product, and AI teams to deliver solutions
  • Improve systems at scale: Lead redesign of workflows, tooling, and processes to improve performance and durability
  • Measure and communicate impact: Define success metrics and communicate performance and recommendations to leadership
  • Elevate the team: Raise the bar through your work and actively mentor others by sharing approaches, guiding problem-solving, and enabling the team to build stronger automation and evaluation capabilities
  • Drive innovation: Stay current on emerging tools and approaches; pilot and translate them into actionable improvements for the team

Minimum Requirements

  • 5+ years of experience and the proven ability to lead complex automation or evaluation initiatives across systems or teams
  • Experience designing, building, and scaling evaluation frameworks, datasets, and quality systems at scale for LLMs or AI products
  • Strong understanding of AI/LLM workflows, automation platforms, and system design
  • Experience in influencing cross-functional stakeholders and aligning priorities
  • Demonstrated ability to define metrics and drive measurable business impact

Preferred Requirements

  • Familiarity with data pipelines, APIs, or production systems
  • Experience mentoring or leading technical contributors
  • Background in AI Operations, applied AI, or process optimization

Compensation

Base salary range: $97,000 - $114,000

Skills

LLMsAI WorkflowsAutomation PlatformsEvaluation FrameworksDatasetsSystem DesignData PipelinesAPIsHitl ProcessesQuality Measurement

Similar roles

ML Engineering jobs
Astera

Senior Software Engineer

AsteraEmeryville, CA

Builds high-performance infrastructure for brain reverse-engineering via large-scale AI models and neuroscience-informed simulations. Architects systems, optimizes RL environments, and scales research prototypes to production.

100k – 400k
HybridML Engineering
Zapier

Sr. Applied AI Engineer

ZapierUnited States

Build and evolve shared AI/ML infrastructure including LLM proxy server, observability tooling, and ML Ops platform capabilities. Focus on LLM Ops and ML Ops to improve how models are accessed, monitored, evaluated, deployed, and governed in production.

102k – 287k
Remote4+ YOEML Engineering
Navan

Senior Software Engineer, AI

NavanNew York, NY

Build agentic AI workflows and LLM-powered loyalty experiences on the Loyalty Wallet team. Requires 6+ years building production AI/LLM systems with strong backend fundamentals.

113k – 252k
On-site6+ YOEML Engineering
Kapitus

Machine Learning Operations (MLOps) Architect - Generative Al Focus

KapitusArlington, VA

Designs and scales enterprise MLOps platform focused on Generative AI and LLMs on AWS, owning architecture for pipelines, deployment, monitoring, governance, and cost optimization. Requires 6+ years ML engineering experience with SageMaker, RAG, and production LLM operationalization.

118k – 189k
On-site6+ YOEML Engineering
Fetch

Senior Automation Specialist, AI Operations

FetchUnited States

Owns evaluation frameworks, datasets, and automations to enhance AI system performance in operations. Requires 3+ years experience with AI tools, LLM workflows, scripting (SQL/Python), and driving ambiguous tasks to execution.

75k – 85k
Remote3+ YOEML Engineering