Model Behavior Engineer
Owns quality for Notion AI products through context engineering, evals, data analysis, debugging, and model evaluation with top AI labs. Requires LLM experience, analytical skills, and driver mentality; no traditional coding.
Automation Lead, AI Operations
Leads automation and evaluation programs for AI Operations, architecting scalable workflows with LLMs, datasets, and quality systems. Requires 5+ years leading complex AI initiatives, cross-functional influence, and metrics-driven impact; familiarity with data pipelines preferred.
97k – 114kUnited StatesML EngineeringRemote5+ YOE
About the role
Role Responsibilities
- Own programs: Lead complex, high-impact automation and evaluation initiatives across workflows or teams
- Design scalable solutions: Architect end-to-end workflows integrating datasets, evaluations, automations, and HITL processes
- Establish standards: Define best practices for datasets, evaluation methodologies, and quality measurement
- Drive cross-functional alignment: Influence priorities and partner with Engineering, Product, and AI teams to deliver solutions
- Improve systems at scale: Lead redesign of workflows, tooling, and processes to improve performance and durability
- Measure and communicate impact: Define success metrics and communicate performance and recommendations to leadership
- Elevate the team: Raise the bar through your work and actively mentor others by sharing approaches, guiding problem-solving, and enabling the team to build stronger automation and evaluation capabilities
- Drive innovation: Stay current on emerging tools and approaches; pilot and translate them into actionable improvements for the team
Minimum Requirements
- 5+ years of experience and the proven ability to lead complex automation or evaluation initiatives across systems or teams
- Experience designing, building, and scaling evaluation frameworks, datasets, and quality systems at scale for LLMs or AI products
- Strong understanding of AI/LLM workflows, automation platforms, and system design
- Experience in influencing cross-functional stakeholders and aligning priorities
- Demonstrated ability to define metrics and drive measurable business impact
Preferred Requirements
- Familiarity with data pipelines, APIs, or production systems
- Experience mentoring or leading technical contributors
- Background in AI Operations, applied AI, or process optimization
Compensation
Base salary range: $97,000 - $114,000
Skills
LLMsAI WorkflowsAutomation PlatformsEvaluation FrameworksDatasetsSystem DesignData PipelinesAPIsHitl ProcessesQuality Measurement
Similar roles
ML Engineering jobsGeneral DiffUSE Job Application
Open general application for computational biology, ML research, data science, software engineering, and program roles at DiffUSE, focused on protein dynamics, structural data infrastructure, and open science tooling.
AI Engineer
As an AI Engineer, you will build and deploy AI-powered solutions to drive business outcomes across Sales, Marketing, and Customer Support. This role requires strong engineering skills, systems thinking, and a product mindset to own initiatives from discovery to deployment.
Software Engineer, Generative AI
Build and scale secure generative AI services and applications using Python, LLMs, and modern frameworks. Own architecture decisions from proposal through production deployment for enterprise customers.