# Software Engineer, Site Reliability

**Company:** [Fal](https://hotfix.jobs/companies/fal)
**Location:** San Francisco, CA
**Role:** DevOps / SRE
**Salary:** $180k – $250k/yr
**Experience:** 5+ years
**Skills:** Kubernetes, Terraform, Ansible, Python, Go, Bash, Prometheus, Grafana, Loki, Thanos, Victoriametrics, Datadog, Fluxcd, Argo CD, Linux Networking
**Posted:** 2026-02-23

> Seasoned SRE owning reliability of Kubernetes-based production infrastructure at scale for a generative AI platform. Responsibilities include operating clusters, CI/CD, SLOs, monitoring, automation with AI, and driving improvements via chaos engineering. Requires 5+ years production experience with deep Kubernetes and observability expertise.

## Job Description

## Key Responsibilities
- Own and operate Kubernetes infrastructure: cluster lifecycle, upgrades, networking, and multi-tenant isolation for customer workloads.
- Build and maintain CI/CD pipelines and deployment infrastructure.
- Leverage AI to automate analysis and resolution of production issues, and improve software development speed, reliability and maintainability.
- Build dashboards, alerting, and anomaly detection across systems.
- Define and enforce SLOs and build out incident response processes.
- Manage and improve networking, load balancing, and service mesh configurations.
- Drive reliability improvements across the stack through automation, runbooks, and chaos engineering.

## Requirements
- 5+ years experience in managing critical production systems and software development workflows.
- Strong production experience setting up and operating Kubernetes at scale, using infrastructure-as-code (Terraform, Ansible).
- Deep knowledge of Linux networking, container networking (CNI plugins, VXLAN, BGP), and DNS.
- Experience building CI/CD systems and GitOps workflows (FluxCD, ArgoCD).
- Proficiency in Python and either Go or Bash for tooling and automation.
- Strong experience with logging, monitoring and alerting (Prometheus, Grafana, Loki, Thanos, VictoriaMetrics, Datadog).
- Excellent communication and ability to drive technical decisions across teams.
- Self-starter who executes quickly, takes ownership, and constantly seeks improvement.

## Nice-to-Haves
- Experience with managing GPU and AI/ML workloads.
- Experience with kernel-based monitoring and routing (eBPF, XDP).
- Experience with security tooling (Falco, Coroot, SIEM).
- Experience with bare metal Kubernetes networking (Calico, Cilium, MetalLB).
- Experience with distributed storage systems (Ceph, Longhorn, etc.).

## Compensation
$180,000-250,000 plus equity + benefits (Range is based across 3 levels Mid, Senior and Staff).

## Similar roles

- [Platform Engineer](https://hotfix.jobs/jobs/14231c8e-abad-4a41-94e3-d41fb2ede6f5) - Govsignals - New York, NY - $180k – $250k/yr
- [Network Engineer, Design & Engineering](https://hotfix.jobs/jobs/48ff3694-5232-4d66-b43c-a091b96c8902) - Fluidstack - New York, NY - $180k – $300k/yr
- [Developer Productivity Engineer](https://hotfix.jobs/jobs/9a60db26-80d5-4b2a-b613-56ae58d49794) - Hightouch - Remote - $180k – $320k/yr
- [Data Center Network Engineer](https://hotfix.jobs/jobs/473ec43a-a5df-4433-99bd-4053f3e6f6bc) - Baseten - San Francisco, CA - $180k – $360k/yr
- [Infrastructure Engineer (Observability)](https://hotfix.jobs/jobs/098c81d9-0827-4baa-a70b-444cc5b04f37) - Lightning AI - Remote - $180k – $200k/yr

**Apply:** https://hotfix.jobs/jobs/e348419f-8436-4eb4-880f-8a936fe4c0f5
**Canonical:** https://hotfix.jobs/jobs/e348419f-8436-4eb4-880f-8a936fe4c0f5