Software Engineer, Backend
Build scalable backend infrastructure for AI observability platform, including real-time data pipelines, LLM integrations with OpenAI/Anthropic/Gemini, and open-source tracing libraries. Requires 5+ years backend experience with Node.js/Go/Python, Postgres/Redis, AWS, and observability tools.
About the job
What you'll do
- Design and ship features that give users deep insight into their LLM usage and performance.
- Integrate, proxy, cache, and collect data from OpenAI, Anthropic, Gemini, and other major model providers that our customers rely on.
- Develop and evolve robust, efficient open source libraries for tracing and evaluating LLM calls inside customer applications.
- Build highly available real-time data pipelines to ingest, store, and query large volumes of structured and semi-structured AI usage data.
- Collaborate closely with product engineers to ship polished, end-to-end features that solve real problems for our customers, such as:
- Bundling and uploading javascript and python code snippets to be used as online, on-demand function calls for LLM workflows.
- Support for parsing and uploading attachments from LLM multimodal outputs into long term storage.
- Role-based access controls and data retention policies for enterprise use cases.
Ideal candidate credentials
- 5+ years of backend engineering experience across multiple parts of the stack
- Strong systems thinking: you’ve owned or contributed to infrastructure projects and know how to design for scale, uptime, and data integrity
- Proficient in modern backend technologies (e.g., Node.js, Go, Python, Rust, Postgres, Redis, Terraform, Docker, AWS)
- Experience with observability tooling (e.g., Grafana, Prometheus, Datadog, or similar) and a passion for making systems operable, available and rock solid.
- Comfortable with ambiguous problems and excited to partner across the company to ship impactful work
- Clear communicator who documents decisions, shares context, and helps lift the team
Skills
Node.js, Go, Python, Rust, Postgres, Redis, Terraform, Docker, AWS, Grafana, Prometheus, Datadog
Similar jobs
Backend Engineering jobs
PrizePicks
United States
Build and scale Ruby on Rails backend services for rewards, incentives, and loyalty features in a high-throughput platform. The role owns complex feature delivery, contributes to architecture, mentors junior engineers, and requires 3+ years of professional software engineering experience.
Twilio
United States
Build and operate backend capabilities for a cloud identity platform, including authentication flows, APIs, and developer experiences. The role requires 3+ years of experience with high-scale production systems and RESTful API development, with Go, TypeScript, and DynamoDB as preferred skills.
Anthropic
San Francisco, CABuild and scale reliable research infrastructure and distributed systems for evolving AI research workflows. The role independently leads complex technical projects, makes foundational architectural decisions, and partners with researchers and engineering teams.
Cloudflare
Austin, TXBuild and operate Internet-scale HTTP and TLS infrastructure, migrate services to a Rust-based proxy, and improve protocol performance. The role requires systems programming experience, strong reliability and security practices, and interest in open-source standards.
Writer
San Francisco, CABuild secure, scalable backend platforms and AI integrations connecting enterprise systems, tools, and data sources. The role requires 5+ years of backend engineering experience, distributed-systems expertise, cloud-native operations, authentication security, and technical leadership.