# Staff Software Engineer, Core Infrastructure

**Company:** [Harvey](https://hotfix.jobs/companies/harvey)
**Location:** New York, NY
**Role:** DevOps / SRE
**Salary:** $201k – $264k/yr
**Experience:** 10+ years
**Skills:** Kubernetes, Terraform, Pulumi, Azure, GCP, Python, Go, Datadog, Sentry, Pagerduty, Redis, CI/CD, Distributed Systems, Observability, Infrastructure As Code
**Posted:** 2026-04-27

> Designs and builds scalable multi-cloud infrastructure powering AI platform, focusing on Kubernetes orchestration, reliability, observability, and operational excellence. Requires 10+ years in infrastructure engineering with deep IaC and cloud expertise.

## Job Description

## What You’ll Do
- Design and build scalable, fault-tolerant infrastructure systems that power Harvey's AI platform across multiple cloud regions
- Own and evolve our multi-cloud infrastructure (Azure, GCP), including Kubernetes orchestration, networking, and container management
- Lead technical initiatives around observability, incident response, and operational excellence — building systems that enable rapid detection and resolution of issues
- Architect and optimize our distributed systems for reliability, including load balancing, quota management, and failover mechanisms
- Partner with Product Engineering and Security teams to ensure our infrastructure is an accelerant, not a constraint
- Drive infrastructure-as-code practices using tools like Terraform and Pulumi to enable reproducible, auditable deployments
- Mentor engineers and raise the technical bar across the organization through code reviews, design reviews, and technical leadership

## Representative Projects
- Design and implement a next-generation model proxy architecture that routes millions of daily inference requests while maintaining model API compatibility and enabling seamless model integration
- Build distributed rate limiting and quota management systems using Redis-backed algorithms to handle bursty traffic patterns without degrading user experience
- Architect multi-region deployment strategies that meet strict data residency requirements for global enterprise customers
- Develop comprehensive observability infrastructure with granular SLA monitoring, burn rate alerts, and detailed token attribution for cost tracking
- Lead the evolution of our CI/CD pipelines to improve developer velocity while maintaining production stability

## What You Have
- 10+ years of experience in Infrastructure Engineering or Platform Engineering in a production environment
- Long track record building and scaling complex, large-scale distributed systems
- Deep proficiency with cloud infrastructure platforms (Azure preferred; GCP or AWS experience transfers well)
- Strong fluency in Infrastructure as Code (IaC) tools — Terraform, Pulumi, or CloudFormation
- Solid understanding of Kubernetes, container orchestration, networking, and cloud security at scale
- Experience with observability tools (Datadog, Sentry) and incident response practices (PagerDuty, Incident.io)
- Strong programming skills in Python, Go, or similar languages
- Excellent problem-solving skills, a \"spidey sense\" of where things could go wrong, and a commitment to operational excellence

## Nice to Have
- Experience building infrastructure for AI/ML workloads or high-throughput inference systems
- Background with distributed rate limiting, load balancing, or quota management systems
- Experience operating multi-tenant platforms with strict security and compliance requirements
- Track record of leading complex cross-functional projects and delivering measurable impact

## Compensation Range
$201,000 - $264,000 USD

## Similar roles

- [Senior Staff Software Engineer, Infrastructure](https://hotfix.jobs/jobs/63a0c07f-752f-423e-812b-2e532ddea9bf) - Rippling - Seattle, WA - $201k – $345k/yr
- [Staff Infrastructure Engineer](https://hotfix.jobs/jobs/75f07283-34de-4989-b193-11c3c3676fd2) - Aurelian - Seattle, WA - $200k – $300k/yr
- [Senior / Staff Platform Engineer](https://hotfix.jobs/jobs/1e15e953-b6f5-46a6-afc4-36a6811e809e) - Radar Labs - New York, NY - $200k – $300k/yr
- [Staff Software Engineer, Infrastructure](https://hotfix.jobs/jobs/ecfe4e28-57ed-4e1d-a9fb-5917d180aeec) - F2 - San Francisco, CA - $200k – $300k/yr
- [Member of Technical Staff, DevOps](https://hotfix.jobs/jobs/c99fed69-8e72-438a-9f3d-3e18894667e7) - Vapi - San Francisco, CA - $200k – $270k/yr

**Apply:** https://hotfix.jobs/jobs/72396864-75e1-4e2d-ae15-cba237b0bec7
**Canonical:** https://hotfix.jobs/jobs/72396864-75e1-4e2d-ae15-cba237b0bec7