# Staff Backend Engineer - Second Horizon

**Company:** [Grafana Labs](https://hotfix.jobs/companies/grafana-labs)
**Location:** Remote
**Role:** Backend Engineering
**Salary:** CA$186k – CA$224k/yr
**Experience:** 7+ years
**Skills:** LLMs, Prompt Engineering, Generative AI, AWS, GCP, Azure, Observability, Mcp, Agent Frameworks, Multi-Agent Workflows, Snowflake, Databricks, API Design, Multi-Tenant Saas, Background Jobs
**Posted:** 2026-08-11

> This Staff Backend Engineer will design and operate production backend services for an AI-native enterprise context system, including ingestion, indexing, retrieval APIs, and agent integrations. The role requires production software experience, cloud-native development, LLM and generative AI expertise, and strong ownership in an ambiguous early-stage environment.

## Job Description

## Responsibilities
- Build, test, and operate backend services for context ingestion, indexing, retrieval orchestration, API access, source configuration, and system administration.
- Define and build a scalable multi-tenant SaaS foundation, including tenant isolation, usage tracking, quotas, audit logs, background jobs, and reliable service boundaries.
- Build APIs and service interfaces for AI agents, MCP tools, CLIs, and internal applications to retrieve context, provenance, confidence signals, and warnings.
- Balance fast experimentation with long-term reliability as the project moves from prototype to production.
- Instrument services with metrics, logs, traces, alerts, and dashboards; use observability tools to improve reliability.
- Contribute to architecture, service boundaries, storage choices, API contracts, deployment patterns, and engineering practices.
- Communicate effectively and collaborate across teams.
- Own AI solutions through scalable, maintainable implementation aligned with user workflows.

## Requirements
- Experience with LLMs, prompt engineering, and applications powered by generative AI.
- Proven experience delivering production software actively used by users.
- Experience in cloud-native environments such as AWS, GCP, or Azure.
- Experience using observability tools to understand and troubleshoot system behavior.
- Strong engineering skills building production-grade, user-facing software systems.
- Familiarity with AI technologies and frameworks, with a practical delivery mindset.
- Ability to iterate quickly, release prototypes, collect feedback, and refine solutions.
- Initiative, ownership, and comfort working through ambiguity.
- Effective collaboration and communication skills.

## Nice to Have
- Experience with agent frameworks or multi-agent workflows.
- Experience as a data analyst or with data platforms such as Looker, Tableau, Power BI, Snowflake, or Databricks.
- Experience building tools for data engineering.

## Compensation and Benefits
- Base compensation in Canada: **CAD $186,368–$223,642**.
- Compensation may vary based on level, experience, and skill set.
- Benefits include equity, bonus where applicable, and other company benefits.
- 100% remote work, global collaboration, career growth opportunities, and an open-source, high-trust culture.

## Similar jobs

- [Staff Backend Engineer - Grafana App Platform](https://hotfix.jobs/jobs/b9fae390-8767-4142-a84f-8ea457c4580f) - Grafana Labs - Remote - CA$186k – CA$224k/yr
- [Staff Software Engineer - Backend](https://hotfix.jobs/jobs/b74325e1-6212-4821-8e15-915d73bb9b02) - Databricks - Vancouver, Canada - CA$173k – CA$238k/yr
- [Senior Staff Software Engineer](https://hotfix.jobs/jobs/3f388646-0ba6-4256-bce9-76ba7aecfede) - Webflow - Remote - $205k – $316k/yr
- [Staff Software Engineer](https://hotfix.jobs/jobs/17cd70e1-50a9-4da8-937a-a0eba74dec8f) - Fivetran - Toronto, Canada - CA$168k – CA$209k/yr
- [Staff Software Engineer, Metadata](https://hotfix.jobs/jobs/16e23b4d-e57c-4aa5-a453-30ec7d50efaa) - Fivetran - Remote - $162k – $199k/yr

**Apply:** https://hotfix.jobs/jobs/de1ec2f7-6312-4e70-b6c0-6fc8476e4f59
**Canonical:** https://hotfix.jobs/jobs/de1ec2f7-6312-4e70-b6c0-6fc8476e4f59