AI Platform Engineer

Responsibilities

• Ship the agent platform to production, including an event-triggered queue, a headless model-agnostic runtime, durable state that survives failed runs, a human review gate, atomic rollback, and complete run logging.
• Own the evaluation layer with golden suites, behavioral assertions, judge criteria with written rubrics, safety cases, and CI gates that block regressions.
• Build and register the agent portfolio for reporting, drafting, linting, triage, question-answering, and a meta layer that observes and improves the platform.
• Enforce governance in code through risk tiers, least-privilege credentials, tool-permission gates, decision audit logs, and autonomy classes where dangerous actions have no code path.
• Design how the system contacts people with interruption budgets, message bundling, and structures that provide value before requesting input.
• Own the platform tooling including the compiler, validator, inventory integrity, and distribution of context and capabilities.
• Compute operating measures from production data, including a pipeline to maturity grades per team.
• Instrument the platform's own return with a ledger that logs absorbed work and computes monthly value.

Requirements

• Shipped production LLM agent systems that other people depended on, with operational history, real users, and incidents (not demos or prompt engineering alone).
• Designed evaluations including golden sets, behavioral assertions, judge rubrics, pass thresholds, and CI gating.
• Deep API work against systems work lives in and authored MCP servers, knowing specific failure modes.
• Owned infrastructure end-to-end in Python on GCP, with cloud warehouse and infrastructure as code; provision, deploy, monitor, rollback, and close architecture decisions.
• Taste in how software contacts humans, treating every notification as spending limited trust.

Nice-to-Haves

• Public work in this space such as an open-source agent framework, MCP server, evaluation harness, or writing on agent reliability.
• LLM observability and cost instrumentation, including tracing runs, attributing spend, and building queries from logs to actionable reports.
• Built an internal platform that non-engineers adopted voluntarily and can describe post-launch changes.