Essential Duties & Responsibilities

Evaluate and recommend AI coding tools (Cursor, Claude Code, GitHub Copilot, Codex CLI, and emerging tools) against Lirio's developer workflows, compliance constraints, and codebase characteristics.
Conduct structured evaluations of new models and tools as they launch, testing against real coding tasks in our environment.
Maintain the evaluation framework and tooling inventory, ensuring the team uses approved, security-reviewed tools on compliance-sensitive systems.

Design and build the agent orchestration layer: instruction files (.cursor/rules/, AGENTS.md, CLAUDE.md), MCP connectors to Azure DevOps and/or GitHub, context packaging templates, and agent routing configurations.
Enable AI coding agents to execute multi-step software development tasks autonomously (decompose, plan, code, test, validate, and submit PRs) with quality gates at each phase and defined escalation points.
Design agent coordination patterns (planner-coder-reviewer, sub-agent delegation) and workflow state management for complex tasks.
Define human escalation triggers for ambiguity, scope boundaries, or compliance-sensitive decisions.
Ensure AI coding agents receive the right context, including project conventions, compliance constraints, coding standards, and relevant codebase context.
Build and maintain work decomposition patterns and templates.
Architect integrations between the AI developer platform and the development ecosystem, including work item tracking, source control, CI/CD pipelines, and code review workflows.

Build rules, instruction files, and CI pipeline checks that flag PHI exposure, tenant isolation concerns, and security issues in AI-generated code.
Translate HIPAA/HITRUST compliance requirements into automated guardrails using defense-in-depth controls.
Define and maintain permission tiers for AI agent operations (read-only, metadata access, code writes with approval).
Ensure AI-assisted delivery produces auditable artifacts, including PR conventions, work item linking, and AI-usage tracking.
Secure the agent input chain by evaluating and mitigating prompt injection risks.

Systematically test new AI models (Claude Opus, GPT Codex, Gemini Pro, etc.) against Lirio's actual coding tasks.
Maintain model routing guidance for task types (complex architecture vs. boilerplate generation vs. test writing vs. code review).
Monitor model quality across updates and pin versions where stability is critical.

Define which types of code changes AI agents can submit with minimal review vs. those requiring full human compliance review.
Establish quality gates and agent supervision practices.
Create and maintain branch/PR conventions for AI-assisted work and work item conventions.

Diagnose root causes of defects, vulnerabilities, or compliance issues in AI-generated code.
Tighten guardrails and adjust workflows based on incident learnings.
Maintain a feedback loop between production issues and the developer platform's safety controls.

Build and maintain observability for the AI developer platform, tracking agent task completion rates, quality gate pass rates, cost per task, guardrail trigger frequency, and model performance trends.