# AI Operations Specialist

**Company:** [Earnin](https://hotfix.jobs/companies/earnin)
**Location:** Mountain View, CA
**Role:** Product Operations
**Salary:** $85k – $100k/yr
**Experience:** 3+ years
**Skills:** Ai Operations, Sop Development, Quality Assurance, APIs, LLMs, Performance Metrics, Dashboards, System Logic, Data Analysis, Customer Support, Product Operations
**Posted:** 2026-09-09

> Operationalize AI agents for customer care by developing procedures, evaluating bot quality, analyzing performance metrics, and driving improvement with Product, Engineering, QA, and vendors. Requires 3–5 years of relevant experience, strong analytical skills, and comfort with APIs and technical documentation.

## Job Description

## Responsibilities

### AI Operating Model and SOP Development
- Design and maintain AI Operating Procedures (AOPs/SOPs) defining Agent AI behavior across supported use cases.
- Translate human-agent judgment into structured system logic, including actions, edge cases, and expected outcomes.
- Inspect and reference APIs, system actions, and integration flows in documentation.
- Update SOPs as product functionality and use cases evolve.

### AI Quality Assurance and Performance Improvement
- Define and execute QA frameworks evaluating containment, accuracy, and resolution quality across vendor bots, internal bots, and LLM-based systems.
- Identify failure modes, escalation drivers, and judgment gaps.
- Partner with Product, Engineering, and vendors to improve AI performance and customer experience.
- Enable and calibrate human QA teams evaluating Agent AI at scale.

### AI Analytics and Operational Insights
- Build and maintain dashboards tracking containment rate, accuracy, escalation rate, and other AI metrics.
- Analyze trends and surface actionable insights for prioritization and iteration.
- Use data to inform readiness decisions and identify performance risks.

### Cross-Functional Execution
- Participate in daily standups with Product and Engineering to align on priorities and iteration loops.
- Provide structured operational feedback to vendors and internal stakeholders.
- Support release-readiness discussions and post-launch performance reviews.
- Travel internationally up to once per quarter to BPO sites supporting AI QA operations.

## Requirements
- College degree or equivalent qualification.
- 3–5 years of relevant experience in operations, QA, product operations, support tooling, or an adjacent field.
- Strong problem-solving, critical-thinking, analytical, interpersonal, organizational, and customer-empathy skills.
- Ability to reason through complex systems and translate human judgment into structured processes.
- Comfort working with APIs, system actions, and technical documentation without being an engineer.
- Experience working with performance metrics and dashboards.
- Ability to lead workstreams involving Product, Engineering, QA, and external partners.
- Willingness to perform execution-focused operational work.

## Compensation
- US base salary: $85,000–$100,000 annually.
- Equity and benefits included.

## Similar jobs

- [Computer Systems Analyst](https://hotfix.jobs/jobs/1a845f0f-2bd1-4042-91a2-2849643923c8) - Sleeper - Seattle, WA - $85k – $108k/yr
- [Legal Operations Project Manager](https://hotfix.jobs/jobs/4390204d-76b5-4ab4-a410-c5b23c8a1da8) - Harvey - New York, NY - $99k – $149k/yr
- [Product Specialist - Concierge](https://hotfix.jobs/jobs/106ac8c0-174e-4589-8ae0-e05ccddd3f81) - Bilt Rewards - New York, NY - $70k – $120k/yr
- [Product Operations Specialist](https://hotfix.jobs/jobs/b1e87e56-93e0-4dc7-b2ce-f5a92bb00fad) - Ramp - New York, NY - $100k – $180k/yr
- [Enablement Operations Manager, Technical Solutions](https://hotfix.jobs/jobs/68cde7f9-d6cc-48ce-97c5-7e4b6e27dff9) - Datadog - New York, NY - $104k – $138k/yr

**Apply:** https://hotfix.jobs/jobs/f1947772-9b9a-4c1b-b5bb-b2e8d8534ed5
**Canonical:** https://hotfix.jobs/jobs/f1947772-9b9a-4c1b-b5bb-b2e8d8534ed5