Leads automation and evaluation programs for AI Operations, architecting scalable workflows with LLMs, datasets, and quality systems. Requires 5+ years leading complex AI initiatives, cross-functional influence, and metrics-driven impact; familiarity with data pipelines preferred.
97k – 114k
Remote5+ YOEML Engineering
About the role
Role Responsibilities
Own programs: Lead complex, high-impact automation and evaluation initiatives across workflows or teams
Establish standards: Define best practices for datasets, evaluation methodologies, and quality measurement
Drive cross-functional alignment: Influence priorities and partner with Engineering, Product, and AI teams to deliver solutions
Improve systems at scale: Lead redesign of workflows, tooling, and processes to improve performance and durability
Measure and communicate impact: Define success metrics and communicate performance and recommendations to leadership
Elevate the team: Raise the bar through your work and actively mentor others by sharing approaches, guiding problem-solving, and enabling the team to build stronger automation and evaluation capabilities
Drive innovation: Stay current on emerging tools and approaches; pilot and translate them into actionable improvements for the team
Minimum Requirements
5+ years of experience and the proven ability to lead complex automation or evaluation initiatives across systems or teams
Experience designing, building, and scaling evaluation frameworks, datasets, and quality systems at scale for LLMs or AI products
Strong understanding of AI/LLM workflows, automation platforms, and system design
Experience in influencing cross-functional stakeholders and aligning priorities
Demonstrated ability to define metrics and drive measurable business impact
Preferred Requirements
Familiarity with data pipelines, APIs, or production systems
Experience mentoring or leading technical contributors
Background in AI Operations, applied AI, or process optimization
Builds high-performance infrastructure for brain reverse-engineering via large-scale AI models and neuroscience-informed simulations. Architects systems, optimizes RL environments, and scales research prototypes to production.
100k – 400k
HybridML Engineering
Sr. Applied AI Engineer
ZapierUnited States
Build and evolve shared AI/ML infrastructure including LLM proxy server, observability tooling, and ML Ops platform capabilities. Focus on LLM Ops and ML Ops to improve how models are accessed, monitored, evaluated, deployed, and governed in production.
102k – 287k
Remote4+ YOEML Engineering
Senior Software Engineer, AI
NavanNew York, NY
Build agentic AI workflows and LLM-powered loyalty experiences on the Loyalty Wallet team. Requires 6+ years building production AI/LLM systems with strong backend fundamentals.
113k – 252k
On-site6+ YOEML Engineering
Machine Learning Operations (MLOps) Architect - Generative Al Focus
KapitusArlington, VA
Designs and scales enterprise MLOps platform focused on Generative AI and LLMs on AWS, owning architecture for pipelines, deployment, monitoring, governance, and cost optimization. Requires 6+ years ML engineering experience with SageMaker, RAG, and production LLM operationalization.
118k – 189k
On-site6+ YOEML Engineering
Senior Automation Specialist, AI Operations
FetchUnited States
Owns evaluation frameworks, datasets, and automations to enhance AI system performance in operations. Requires 3+ years experience with AI tools, LLM workflows, scripting (SQL/Python), and driving ambiguous tasks to execution.