Site Reliability Engineer

What You'll Do

• Build and orchestrate API-based products from the ground up. Design and implement low-latency, high-reliability APIs for leaderboards, models, and arenas.
• Ship enterprise-grade infrastructure: rate limiting, authentication, usage metering, cost attribution, audit logging, and SOC 2 compliance.
• Build deep observability with distributed tracing, latency breakdowns, token-level usage tracking, and real-time dashboards.
• Build AI-centered products by integrating with the core evaluation platform, Arena data, and customer-specific benchmarks. Collaborate with the research team to turn novel ideas into full-featured products.

What We're Looking For

• 6+ years of backend engineering experience, with meaningful time spent on distributed systems, infrastructure, or developer-facing platforms.
• Strong proficiency in Go and/or Rust, with hands-on experience building high-throughput APIs or proxy/gateway systems.
• Experience with LLM provider APIs (OpenAI, Anthropic, Google, etc.) and understanding of challenges like streaming, token management, rate limits, model-specific quirks.
• Solid cloud infrastructure skills with AWS or GCP, Kubernetes, Terraform, and database systems like Postgres and Redis.
• A product-oriented mindset focused on developer experience.
• Comfort with ambiguity in a startup environment.

Nice to Have

• Experience building API gateways, proxies, or developer tools (Bifrost, Kong, Envoy, Tyk, or custom).
• Background in ML infrastructure, model serving, or evaluation frameworks.
• Experience building enterprise-ready features: SSO, RBAC, audit logs, multi-tenancy.
• Experience building billing infrastructure around Stripe, Metronome and Orb.
• Familiarity with the modern AI infra stack (vLLM, LiteLLM, LangChain, etc.).