# Staff Software Engineer, Observability & Profiling

**Company:** [Anthropic](https://hotfix.jobs/companies/anthropic)
**Location:** London, United Kingdom
**Role:** DevOps / SRE
**Salary:** £325k – £390k/yr
**Experience:** 10+ years
**Skills:** Observability, Monitoring, Telemetry Pipelines, Metrics, Logging, Distributed Tracing, Continuous Profiling, Ebpf, OpenTelemetry, Kernel Debugging, Syscalls, Performance Engineering, Ai/Llms, Accelerator Workloads, Telemetry Storage
**Posted:** 2026-09-02

> Build and operate foundational observability infrastructure spanning telemetry pipelines, profiling, tracing, and diagnostic tooling across large-scale compute clusters. The role requires deep systems-level experience and 10+ years of relevant industry experience.

## Job Description

## Responsibilities
- Design and build scalable telemetry ingest and storage pipelines for metrics, logs, traces, and error data across multi-cluster infrastructure.
- Build low-overhead observability solutions providing deep fleet-wide visibility into system behavior.
- Own and evolve core observability platforms, including migrations and architectural improvements that improve reliability, reduce cost, and scale.
- Build instrumentation libraries, SDKs, and eBPF-based auto-instrumentation with and without code changes.
- Reduce mean time to detection and resolution through cross-signal correlation, unified query interfaces, and AI-assisted diagnostic tooling.
- Use continuous profiling and utilization telemetry to optimize CPU, memory, and accelerator fleets.
- Partner with Research, Inference, Product, and Infrastructure teams to meet their operational visibility needs.

## Requirements
- Hands-on experience building and operating large-scale observability or monitoring infrastructure.
- Deep end-to-end experience with observability signals, from instrumentation through ingest, query, and analysis.
- Understanding of high-throughput telemetry pipelines and operational-data collection, storage, and querying tradeoffs at scale.
- Ability to troubleshoot below the application layer, including the kernel, network stack, or hardware.
- Excellent communication and collaboration skills.
- Bachelor’s degree or equivalent combination of education, training, and experience.

## Nice-to-haves
- 10+ years of relevant industry experience, including large-scale observability or monitoring infrastructure.
- Production experience with eBPF-based tracing, profiling, or network visibility.
- Experience running continuous profiling at fleet scale, including overhead budgets and symbolization.
- Kernel- and syscall-level debugging and performance engineering experience.
- Experience profiling or instrumenting accelerator workloads.
- Experience with high-cardinality metrics systems or large-scale telemetry storage backends.
- Experience with OpenTelemetry instrumentation, collector pipelines, and tail-based sampling.
- Interest in applying AI or LLMs to root-cause analysis, anomaly detection, or intelligent alerting.

## Compensation
- Annual salary: £325,000–£390,000 GBP.
- Benefits include competitive compensation, equity donation matching, generous vacation and parental leave, flexible working hours, and office collaboration space.

## Similar jobs

- [Staff Software Engineer, AI Reliability Engineering](https://hotfix.jobs/jobs/66b246fd-6b0e-4283-b1ff-c460780b61a2) - Anthropic - London, United Kingdom - £325k – £390k/yr
- [Senior/Staff Kubernetes Infrastructure Engineer](https://hotfix.jobs/jobs/5276ef82-8104-4269-8e3f-7f0e02d35c2d) - Fal - Remote - $180k – $250k/yr
- [Staff Engineer, Platform & Infrastructure](https://hotfix.jobs/jobs/48b953ba-a6fb-4833-9ad0-140dfe4943dc) - Nango - Remote - $140k – $220k/yr
- [Staff Platform Engineer](https://hotfix.jobs/jobs/37cd4cd2-d007-4013-8153-5443ae70f1dd) - Nango - Remote - $140k – $220k/yr
- [Site Reliability Engineer, Intermediate to Senior Staff](https://hotfix.jobs/jobs/cb4f85a5-5a0b-4be9-a1ab-cd4b0c01d37b) - GitLab - Remote - $126k – $314k/yr

**Apply:** https://hotfix.jobs/jobs/be415182-687b-4ae9-a1c2-4b70b877514b
**Canonical:** https://hotfix.jobs/jobs/be415182-687b-4ae9-a1c2-4b70b877514b