# Staff Backend Engineer - Grafana Second Horizon

**Company:** [Grafana Labs](https://hotfix.jobs/companies/grafana-labs)
**Location:** Remote
**Role:** Backend Engineering
**Salary:** €94k – €113k/yr
**Experience:** 7+ years
**Skills:** Python, LLMs, Prompt Engineering, Generative AI, Agent Frameworks, Mcp, AWS, GCP, Microsoft Azure, Multi-Tenancy, REST APIs, Observability, Metrics, Distributed Systems, Snowflake
**Posted:** 2026-07-23

> Build and operate production backend services for an AI-native enterprise context layer, including ingestion, storage, retrieval APIs, and agent integrations. The role requires staff-level architectural ownership, cloud-native experience, LLM and GenAI application expertise, and comfort working autonomously in an ambiguous early-stage environment.

## Job Description

## Responsibilities
- Design, implement, test, and operate backend services for context ingestion, indexing, retrieval orchestration, API access, source configuration, and system administration.
- Define and build a scalable multi-tenant SaaS foundation, including tenant isolation, usage tracking, quotas, audit logs, background jobs, and reliable service boundaries.
- Build APIs and service interfaces for AI agents, MCP tools, CLIs, and internal applications to retrieve context, provenance, confidence signals, and warnings.
- Balance rapid experimentation with long-term reliability as the project progresses from prototype to production.
- Instrument services with metrics, logs, traces, alerts, and dashboards; use observability tools to understand system behavior and improve reliability.
- Shape architecture, service boundaries, storage choices, API contracts, deployment patterns, and engineering practices.
- Communicate effectively and contribute across teams.
- Own AI solutions through scalable, maintainable implementation aligned with user workflows.

## Requirements
- Strong experience building production-grade, user-facing software systems.
- Familiarity with AI technologies and frameworks, with a practical delivery mindset.
- Experience with rapid prototyping, experimentation, feedback collection, and iteration.
- Ability to take initiative, define scope in ambiguous situations, and drive projects forward.
- Strong collaboration and communication skills.
- Experience with LLMs, prompt engineering, and GenAI-powered applications.
- Proven experience delivering software to production and active users.
- Experience in cloud-native environments such as AWS, GCP, or Azure.
- Experience using observability tools to understand and troubleshoot system behavior.

## Nice-to-haves
- Experience with agent frameworks or multi-agent workflows.
- Experience as a data analyst or with data platforms such as Looker, Tableau, Power BI, Snowflake, or Databricks.
- Experience building tools for data engineering.

## Compensation and Benefits
- Spain base compensation range: **EUR 94,025–112,830**.
- Restricted Stock Units (RSUs) are included with all roles.
- Compensation ranges and benefits are country-specific.

## Similar jobs

- [Staff Software Engineer, GRC](https://hotfix.jobs/jobs/3a506b48-97b0-44a6-8c94-1b8de6be5ad9) - Vanta - Remote - $260k – $306k/yr
- [Staff Software Engineer](https://hotfix.jobs/jobs/1c1c4d74-9822-4ad9-84fe-efdf89104398) - Anaconda - Remote
- [Staff Software Engineer - UI Platform](https://hotfix.jobs/jobs/200d452a-4463-48dc-9a41-38a30ef8e88e) - Stripe - Barcelona, Spain
- [Staff Engineer - Data Platform Experience](https://hotfix.jobs/jobs/83c4a3b7-0151-42a8-8538-2949b6b83346) - Datadog - Lisbon, Portugal
- [Staff Engineer](https://hotfix.jobs/jobs/354d25d7-876c-4626-bf3e-7563e8a7cffd) - MongoDB - Dublin, Ireland

**Apply:** https://hotfix.jobs/jobs/908f3552-f903-42e0-b9f3-353b6d3fa736
**Canonical:** https://hotfix.jobs/jobs/908f3552-f903-42e0-b9f3-353b6d3fa736