Staff Software Engineer - Databases, Tempo
Leads architecture and delivery of complex distributed data-system initiatives for Tempo, Grafana’s open-source tracing backend. The role requires deep production systems experience, strong software craftsmanship, operational ownership, and technical leadership across multiple teams.
About the job
Responsibilities
- Lead multi-quarter technical initiatives from problem framing through rollout, including trace aggregation APIs, autoscaling cells, customer limits, and query engine improvements.
- Own the architecture of core Tempo components: ingestion, storage, query, and metrics generation.
- Drive design reviews and make trade-offs across performance, cost, and complexity; document architectural decisions.
- Design structured, deterministic, discoverable APIs for humans, agents, downstream products, and external integrators.
- Drive operational excellence through SLO ownership, automation, parameterized rollouts, actionable alerts, and toil reduction.
- Partner with Product and sibling teams to understand and unblock downstream product needs.
- Mentor engineers through code review, design feedback, pairing, and technical writing.
- Participate in on-call, incident response, and post-incident learning.
- Contribute to the open-source Tempo project, review external contributions, and engage the community.
Requirements
- Track record of leading complex, multi-quarter initiatives spanning design, delivery, and operations.
- Substantial hands-on experience building and operating distributed data systems in production, such as ingestion pipelines, storage engines, or query execution systems.
- Strong software craftsmanship and experience writing clean, robust, performant, maintainable software.
- Strong Go experience, or deep experience in Rust, C, or C++ with a path to Go.
- Experience owning production services, carrying a pager, reducing toil, and working with SLOs.
- Customer-focused and pragmatic approach with short feedback loops, MVP delivery, and iterative improvement.
- Clear communication through design documents, reviews, and shipped code in a fully remote, asynchronous environment.
Nice-to-haves
- Experience with tracing, OpenTelemetry, or large-scale observability systems.
- Experience designing query languages, SQL- or TraceQL-like engines, or programmatic APIs for services and agents.
- Experience with columnar storage formats such as Parquet or purpose-built on-disk analytical formats.
- Experience operating multi-tenant, multi-cell SaaS infrastructure at scale on Kubernetes.
- Experience building systems for AI and large language model consumers.
Compensation and Benefits
- Company-funded usage budget for AI coding assistants.
- Access to frontier AI models, including GPT-5.6, Claude Fable, and Gemini Pro 3.1.
Skills
Go, Rust, C, C++, Distributed Systems, Ingestion Pipelines, Storage Engines, Query Execution, Traceql, OpenTelemetry, Parquet, Kubernetes, SLOs, Ai Assistants, LLMs
Similar jobs
Backend Engineering jobs
Anaconda
United StatesStaff Software Engineer leading the production toolchain for compiling and distributing complex Python packages across Linux, macOS, and Windows. Requires 10+ years of software development experience and deep C/C++ expertise in compilation, linking, architecture, and technical strategy.
GitLab
Bengaluru, India
Provides cross-team technical leadership for large backend initiatives, modular architecture, production operations, and AI-assisted engineering adoption. The role requires deep backend architecture experience, monolith modernization expertise, strong written influence, and mentoring ability.
Addepar
Edinburgh, United Kingdom
Leads the design and implementation of resilient backend API services and distributed platform architecture for a financial data ecosystem. The role requires extensive software development experience, expert API and systems design knowledge, and strong technical leadership and mentoring.
Shield AI
London, United Kingdom
Leads high-performance C++ development, real-time modeling and simulation, and integration of third-party software and hardware systems. Requires staff-level engineering experience, strong systems architecture expertise, and leadership of complex technical projects.
Nango
United StatesStaff Backend Software Engineer responsible for designing and scaling developer infrastructure, APIs, and backend services while shaping product direction and collaborating directly with customers. Requires extensive backend experience, Node.js proficiency, and a strong focus on developer experience.