# Staff Software Engineer, Core Infrastructure

**Company:** [Harvey](https://hotfix.jobs/companies/harvey)
**Location:** San Francisco, CA
**Role:** DevOps / SRE
**Salary:** $236k – $290k/yr
**Experience:** 10+ years
**Skills:** Kubernetes, Terraform, Pulumi, Azure, GCP, Datadog, Sentry, Pagerduty, Python, Go, Redis, CI/CD
**Posted:** 2026-04-27

> Designs and builds scalable multi-cloud infrastructure (Azure, GCP) powering AI platform, including Kubernetes orchestration, observability, and distributed systems for reliability at massive scale. Requires 10+ years in infrastructure engineering with IaC expertise.

## Job Description

## What You’ll Do
- Design and build scalable, fault-tolerant infrastructure systems that power Harvey's AI platform across multiple cloud regions
- Own and evolve our multi-cloud infrastructure (Azure, GCP), including Kubernetes orchestration, networking, and container management
- Lead technical initiatives around observability, incident response, and operational excellence — building systems that enable rapid detection and resolution of issues
- Architect and optimize our distributed systems for reliability, including load balancing, quota management, and failover mechanisms
- Partner with Product Engineering and Security teams to ensure our infrastructure is an accelerant, not a constraint
- Drive infrastructure-as-code practices using tools like Terraform and Pulumi to enable reproducible, auditable deployments
- Mentor engineers and raise the technical bar across the organization through code reviews, design reviews, and technical leadership

## Representative Projects
- Design and implement a next-generation model proxy architecture that routes millions of daily inference requests while maintaining model API compatibility and enabling seamless model integration
- Build distributed rate limiting and quota management systems using Redis-backed algorithms to handle bursty traffic patterns without degrading user experience
- Architect multi-region deployment strategies that meet strict data residency requirements for global enterprise customers
- Develop comprehensive observability infrastructure with granular SLA monitoring, burn rate alerts, and detailed token attribution for cost tracking
- Lead the evolution of our CI/CD pipelines to improve developer velocity while maintaining production stability

## What You Have
- 10+ years of experience in Infrastructure Engineering or Platform Engineering in a production environment
- Long track record building and scaling complex, large-scale distributed systems
- Deep proficiency with cloud infrastructure platforms (Azure preferred; GCP or AWS experience transfers well)
- Strong fluency in Infrastructure as Code (IaC) tools — Terraform, Pulumi, or CloudFormation
- Solid understanding of Kubernetes, container orchestration, networking, and cloud security at scale
- Experience with observability tools (Datadog, Sentry) and incident response practices (PagerDuty, Incident.io)
- Strong programming skills in Python, Go, or similar languages
- Excellent problem-solving skills, a \"spidey sense\" of where things could go wrong, and a commitment to operational excellence

## Nice to Have
- Experience building infrastructure for AI/ML workloads or high-throughput inference systems
- Background with distributed rate limiting, load balancing, or quota management systems
- Experience operating multi-tenant platforms with strict security and compliance requirements
- Track record of leading complex cross-functional projects and delivering measurable impact

## Compensation Range
$236,000 - $290,000 USD

## Similar roles

- [Staff Software Engineer - Forge](https://hotfix.jobs/jobs/1aa46952-edc5-4cbc-9c3f-b39f8144851e) - Snowflake - Bellevue, WA - $236k – $339k/yr
- [Staff Software Engineer - Container Platform](https://hotfix.jobs/jobs/50c12723-254d-46da-b7ab-6bd520c3b896) - Snowflake - Menlo Park, CA - $236k – $339k/yr
- [Sr. Staff Software Engineer, Observability](https://hotfix.jobs/jobs/a97725e8-186d-4fd0-aea6-7a8ac593c2b5) - Databricks - Mountain View, CA - $235k – $295k/yr
- [Senior Staff Software Engineer, Managed Orchestration](https://hotfix.jobs/jobs/13165fc9-83b3-4e03-af13-4661f44f9bce) - Crusoe - San Francisco, CA - $238k – $288k/yr
- [Staff Software Engineer, Core Platform](https://hotfix.jobs/jobs/90234f16-d5e2-4066-a43a-884905d027a9) - Vanta - Remote - $238k – $280k/yr

**Apply:** https://hotfix.jobs/jobs/ed02606b-8ac5-4027-b2a3-f70767a67a50
**Canonical:** https://hotfix.jobs/jobs/ed02606b-8ac5-4027-b2a3-f70767a67a50