Staff Software Engineer, Alerting Platform

Responsibilities

• Set the architecture for evaluating rule-based thresholds, ML anomaly scores, and correlation logic to produce high-fidelity alerts.
• Own the reliability of the alerting pipeline end to end, including OpenSearch alert evaluation, Kafka delivery, callbacks, downstream notification, idempotency, and sustained-load behavior.
• Correlate security tooling, monitoring telemetry, syslog, OpenTelemetry, and L2–L4 network protocols into network and system topologies and dependency maps.
• Make topology context usable for LLM-based investigation, troubleshooting, correctness, and remediation workflows.
• Establish the technical foundation for generating alert definitions from normalized data models using LLMs.
• Provide architectural guidance across Command|Alert teams and workstreams.
• Balance new correlation and detection capabilities with long-term scalability and reliability.
• Mentor engineers and represent the technical direction to external stakeholders.

Requirements

• Production experience designing, building, or operating high-reliability alerting or notification systems at scale.
• Experience running rule-based and ML-based detection logic against real traffic.
• Strong Kafka experience, including producing and consuming event streams for downstream delivery.
• Command of security tooling output, monitoring telemetry, syslog, OpenTelemetry, NetFlow or sFlow, SNMP, ICMP, and firewall logs.
• Experience building systems with reliable webhooks, idempotency, and delivery guarantees under load.
• Recognized mastery of Go and/or Python.
• Experience with container orchestration and multi-cloud environments.
• Ability to make organization-level architecture and technology decisions.

Nice to Have

• Temporal or comparable workflow orchestration experience.
• Experience using LLMs with structured system or network data.
• Familiarity with Flink, Protocol Buffers, Argo, or Spacelift.
• Familiarity with Memgraph or another graph store.
• Experience with osquery, Steampipe, or similar endpoint and cloud inventory tooling.
• Experience in multi-tenant, cloud-native environments with secrets management and TLS at scale.

Benefits

• Room to grow at a high-growth company.
• Innovative environment with meaningful impact.
• Flexible time off.
• Fun events.
• Employee referral bonuses.