Lead Automations Analyst
Owns the end-to-end design, build, deployment, and optimization of AI workflows inside ORA for Customer Success, spanning call preparation, guidance, grading, and follow-up. Requires 7+ years in applied AI or related solution delivery, strong LLM and data-flow expertise, and the ability to translate business problems into shipped systems.
About the job
Responsibilities
- Own end-to-end AI solution delivery inside ORA, from problem definition through design, build, deployment, monitoring, and iteration.
- Translate Customer Success problems into AI solution designs, including agent behavior, context, success criteria, orchestration, skills, models, and knowledge sources.
- Build orchestrators and sub-agents; author prompts, skills, and knowledge-base entries.
- Design measurable feedback loops using logs, evaluation signals, and advisor feedback.
- Design AI workflows for pre-call briefs, in-call guidance, post-call grading and summaries, transcription-driven follow-ups, and internal knowledge tools.
- Partner with RevOps to define platform capability gaps, requirements, acceptance criteria, edge cases, and customer impact.
- Integrate new ORA capabilities and validate outcomes.
- Work with Quality Analyst II and Success leadership to prioritize fixes and builds based on evaluation data and business impact.
- Define standards, playbooks, testing practices, observability, model-selection approaches, orchestration patterns, and prompt architecture.
- Experiment with emerging models, techniques, and evaluation approaches and share findings.
Requirements
- 7+ years in AI/ML solution work, technical product, prompt engineering, applied AI, automation engineering, or a closely related end-to-end delivery role.
- Strong understanding of LLM behavior, prompt design, orchestration patterns, model-selection trade-offs, failure modes, and evaluation design.
- Strong understanding of databases and data flows between systems.
- Ability to read code, understand ORA behavior, debug integrations, and collaborate technically with RevOps engineers; production coding is not required.
- Ability to scope problems, design solutions, write requirements and acceptance criteria, and ship end to end.
- Strong written and verbal communication skills.
- End-to-end ownership mindset.
- Bachelor’s degree in a related field or equivalent practical experience.
Preferred Qualifications
- Experience with AI development platforms such as Claude, Cursor, or similar tools.
- Experience designing orchestration patterns, sub-agent workflows, or multi-step AI solutions.
- Familiarity with retrieval-augmented generation, vector databases, or knowledge-base architecture.
- Familiarity with SQL or similar database querying.
- Experience with APIs, webhooks, or data integration between systems.
- Background in customer success, support, sales enablement, or call-center operations.
Skills
Artificial Intelligence, Machine Learning, LLMs, Prompt Engineering, Ai Orchestration, Model Selection, Evaluation Design, Databases, SQL, APIs, Webhooks, Retrieval-Augmented Generation, Vector Databases, Knowledge Bases, Claude
Similar jobs
AI Research jobs
Decagon
San Francisco, CAResearch and build safety models, evaluations, and runtime safeguards for conversational AI agents, addressing prompt injection, unsafe tool use, privacy, and policy risks. Requires 4+ years in AI/ML engineering, research, or safety plus experience deploying and evaluating language models or agentic systems.
Function Health
United States
Evaluates and improves AI-generated clinical outputs, partnering with product and engineering teams to establish safety, accuracy, and clinical-quality standards. Requires an MD, DO, or equivalent clinical doctorate, substantial patient-care experience, strong clinical judgment, and the ability to learn AI evaluation techniques.
Gusto
Denver, COOwns reusable patterns, standards, and tooling for production agentic service workflows, guiding platform priorities, automation measurement, and quality governance. Requires 8+ years in operations, product, or AI, hands-on agentic workflow experience, and strong LLM, metrics, and cross-functional influence skills.
Anthropic
San Francisco, CAApplied AI Research Engineer who tests model capabilities, builds demos and evaluations, supports strategic customer implementations, and translates field insights into product and research direction. Requires 6+ years of technical experience, programming proficiency, LLM development experience, and strong communication skills.
Ai2
Seattle, WA
Conduct research and build open foundation models and training systems aimed at accelerating scientific discovery. The role requires a PhD-level background and substantial experience training foundation models, with expertise in agentic training or multimodal data preferred.