Senior Software Engineer, Monitoring & Alerting
200k – 288kMenlo Park, CAOnsite5+ YOE
Summary
Builds and scales backend systems for monitoring, alerting, and anomaly detection in an AI-powered observability platform. Requires 5+ years backend experience with high-concurrency SQL, distributed architectures, and end-to-end feature ownership.
About the role
Responsibilities
- Drive high-impact features end-to-end: from anomaly detection and compound monitor logic to simple alerting workflows, owning scope, architecture, and delivery.
- Design and scale the monitoring and alerting pipeline to support high concurrency and high-throughput SQL workloads, ensuring reliability for hundreds of terabytes of telemetry processed daily.
- Own the full product surface of the monitoring system, iterating on backend behavior, AI agent interactions, and pipeline internals based on direct customer feedback.
- Build AI-native workflows for monitor creation and alert triage, bringing intelligence to every step of the detection-to-resolution lifecycle.
- Collaborate cross-functionally with forward deployment engineers, product managers, designers, support engineers, and sales engineers to solve real customer reliability problems.
- Mentor junior engineers, lead technical design decisions, and contribute to engineering excellence across the team.
- Work across the backend stack as needed: persistent queues, event-driven architectures, and high-concurrency SQL systems.
Requirements
- 5+ years of backend development experience, with a strong record of shipping reliable, high-scale systems (or equivalent experience).
- Deep understanding of high-concurrency and high-scalability system design, including distributed workloads, persistent queues, and fault-tolerant architectures.
- Strong programming and debugging skills, with the ability to navigate complex systems under pressure.
- A product mindset: seek to understand user motivation and use cases before diving into technical solutions.
- Demonstrated experience owning and delivering medium-sized projects end-to-end with minimal guidance.
- Experience mentoring junior engineers and leading technical design decisions.
- BS in Computer Science or equivalent professional experience.
Nice-to-Haves
- Prior experience building or working on observability, monitoring, or alerting systems.
- Familiarity with AI/ML-driven workflows such as anomaly detection, AI-assisted tooling, or intelligent automation.
- Experience with large-scale data platforms including Snowflake, data lakes, or streaming systems.
- Hands-on experience with high-concurrency SQL workloads or query optimization.
- Background in event-driven architectures, distributed queues, or streaming pipelines.
- Prior experience in a high-growth startup or fast-paced product environment.
Skills
SQLpersistent queuesevent-driven architecturesdistributed systemsanomaly detectionAI/ML workflowsSnowflakestreaming systemsobservabilitymonitoring systems
Similar roles at this salary range
All Backend Engineering jobs →Staff Software Engineer, Growth AI
Staff Software Engineer anchoring AI-powered growth products across SEO and exploratory teams. Architect production ML systems, partner with ML orgs, and set technical direction as a senior IC.
208k – 365kSan Francisco, CA +3Backend EngineeringHybridJavaLLMs
Staff Backend Engineer
Staff Backend Engineer to architect and scale voice AI systems for 911 centers, optimizing realtime pipelines and integrating with customer systems. Requires 6+ years of experience and strong engineering fundamentals.
180k – 240kSeattle, WABackend EngineeringOn-siteGoSQL