Site Reliability Engineer

Build and operate the scalable, low-latency infrastructure powering Arena's real-world AI model evaluation platform, including API gateways, observability, and enterprise features for frontier model routing and evaluation.

About the job

What You'll Do

Build and orchestrate API-based products from the ground up. Design and implement low-latency, high-reliability APIs for leaderboards, models, and arenas.
Ship enterprise-grade infrastructure: rate limiting, authentication, usage metering, cost attribution, audit logging, and SOC 2 compliance.
Build deep observability with distributed tracing, latency breakdowns, token-level usage tracking, and real-time dashboards.
Build AI-centered products by integrating with the core evaluation platform, Arena data, and customer-specific benchmarks. Collaborate with the research team to turn novel ideas into full-featured products.

What We're Looking For

6+ years of backend engineering experience, with meaningful time spent on distributed systems, infrastructure, or developer-facing platforms.
Strong proficiency in Go and/or Rust, with hands-on experience building high-throughput APIs or proxy/gateway systems.
Experience with LLM provider APIs (OpenAI, Anthropic, Google, etc.) and understanding of challenges like streaming, token management, rate limits, model-specific quirks.
Solid cloud infrastructure skills with AWS or GCP, Kubernetes, Terraform, and database systems like Postgres and Redis.
A product-oriented mindset focused on developer experience.
Comfort with ambiguity in a startup environment.

Nice to Have

Experience building API gateways, proxies, or developer tools (Bifrost, Kong, Envoy, Tyk, or custom).
Background in ML infrastructure, model serving, or evaluation frameworks.
Experience building enterprise-ready features: SSO, RBAC, audit logs, multi-tenancy.
Experience building billing infrastructure around Stripe, Metronome and Orb.
Familiarity with the modern AI infra stack (vLLM, LiteLLM, LangChain, etc.).