AI Operations Engineer
Build and operate AI-driven workflows, integrations, model-routing systems, and agent infrastructure across business and engineering functions. The role also supports model evaluation, AI cost optimization, governance safeguards, and organization-wide training.
About the job
Responsibilities
Build AI-Powered Solutions
- Develop and deploy solutions using LLMs, automation frameworks, and internal tooling for business and engineering use cases.
- Integrate AI into HRIS, ATS, ERP, procurement, ticketing, CI/CD, developer tooling, CLM, and matter-management systems.
- Build workflows for document generation, data extraction, knowledge retrieval, and decision support.
- Use modern integration standards such as MCP to provide AI models and agents with secure access to internal knowledge and context.
Support Model Evaluation
- Test and benchmark AI models for accuracy, latency, cost, and safety.
- Support model risk assessments and due diligence with Security and GRC.
- Help implement and maintain internal model routing.
Support Cost Visibility
- Build dashboards and reporting for AI and LLM spend.
- Implement tagging, monitoring, and alerting for cost anomalies.
- Identify and execute cost-optimization opportunities, including prompt efficiency, caching, and model right-sizing.
Maintain Agent Infrastructure
- Provision, credential, and de-provision AI agent access.
- Support deployment, versioning, monitoring, and retirement of agents.
- Maintain and patch agent infrastructure and dependencies.
- Maintain audit trails and logging for agent actions.
Training and Enablement
- Design and deliver AI training for technical and non-technical audiences.
- Create playbooks, guides, templates, and reusable components for safe AI adoption.
- Run onboarding sessions, office hours, and workshops.
- Establish feedback loops to improve training, tooling, and documentation.
Requirements
- Experience building with modern AI and machine-learning tools and frameworks.
- Experience building and consuming APIs, developing data pipelines, and architecting system integrations.
- Working knowledge of cloud infrastructure and experience hosting or operating production services.
- Experience comparing and evaluating AI models across providers, including cost, latency, and quality trade-offs.
- Experience using billing or usage data for cost visibility, forecasting, or optimization.
- Ability to translate business and engineering workflows into technical solutions.
- Experience integrating business systems and engineering or developer tooling.
- Experience creating training materials, documentation, or workshops for mixed audiences.
- Understanding of data privacy, security, and compliance considerations.
- Experience implementing AI safeguards, including identity and access controls for automated or non-human actors.
Compensation and Benefits
- ATS-listed salary: $170,000–$250,000 USD.
- Typical US starting salary: $170,000–$220,000 USD.
- Typical US premium-market starting salary: $200,000–$250,000 USD.
- Healthcare contributions.
- Company stock options.
- Flexible time off in the US and generous entitlement in other countries.
- $500 home-office setup benefit for remote employees.
- Opportunities to attend company-wide offsites.
- Flexible, remote-friendly work environment.
Skills
LLMs, Ai/Ml Frameworks, APIs, Data Pipelines, AWS, GCP, Microsoft Azure, Mcp, Model Evaluation, Model Routing, Cost Optimization, Identity And Access Management, CI/CD, HRIS, ERP
Similar jobs
ML Engineering jobs
Clay
New York, NY
Build and ship production AI agents and the platform infrastructure that makes them reliable, steerable, and measurable. The role requires strong backend fundamentals, production LLM or agent experience, and expertise in evaluations, retrieval, orchestration, or tool-use design.
Clay
San Francisco, CA
Build and ship production machine-learning systems that learn from customer data and behavior, including recommendations, LLM-powered features, evaluation systems, and ML infrastructure. The role requires 5+ years of ML engineering or ML-heavy software engineering experience and strong production systems expertise.
Mirage
New York, NY
Build and deploy agentic systems that power AI-driven creative video workflows. The role requires 5+ years of experience, production ML or agentic pipeline development, context engineering, and expertise in evaluation and agent infrastructure.
Mirage
New York, NY
Build and advance agentic machine-learning systems for multimodal creative tasks, with a focus on video understanding, reasoning, control, and tool use. The role requires strong production ML or agent-pipeline experience and deep knowledge of modern LLM techniques.
Taste Labs
San Francisco, CA
Build evaluation methods, RL environments, agent tooling, and scalable infrastructure that make subjective qualities such as design and taste measurable for frontier AI models. The role requires experience with evaluations, RL environments, ML or post-training, plus strong backend engineering skills.