Responsibilities
- Build and maintain dashboards tracking performance across all live AI programs (automation rate, containment, AHT, CSAT, model quality) using Zendesk Explore, Looker, or similar.
- Identify performance degradation, intent gaps, hallucination, model drift, and guardrail failures across Zendesk AI, chatbots, ticketbots, and automation workflows.
- Own incident response process for AI quality issues: triage, escalate, drive resolution with post-mortem.
- Serve as primary admin for net-new AI tooling post-launch (VoiceAI, Zendesk AI agents).
- Maintain active optimization backlogs for launched programs.
- Drive post-launch iteration with Chatbot Strategy Manager and tooling teams.
- Own AI program documentation library: runbooks, flow documentation, QA procedures, training guides.
- Maintain records of AI program behavior, guardrails, and change history for compliance.
- Partner with CX AI & Automation Manager during launch phases.
- Build feedback loops from AI performance data to inform leadership decisions.
Requirements
- Hands-on experience analyzing performance data from AI or automation tools in a support environment.
- Understanding of AI failure modes (hallucination, model drift, guardrail gaps, intent misclassification) and designing QA processes.
- Proficiency with data visualization tools (Zendesk Explore, Looker, Tableau) and writing queries.
- Strong documentation skills.
- Analytical, detail-oriented, comfortable with dashboards and QA analysis.
- Experience with Zendesk (or comparable) and AI tools (Zendesk AI, Decagon).
- Experience in fintech, financial services, or regulated industry.
- Continuous improvement mindset and systems thinking.
Compensation
Base salary: $110,000 - $140,000 (USD). Competitive equity package, health/dental/vision benefits, 401(k) with employer match, generous PTO.