# Staff Software Engineer - Test Systems & Tooling

**Company:** [Temporal](https://hotfix.jobs/companies/temporal)
**Location:** Remote
**Role:** QA Engineering
**Salary:** $212k – $278k/yr
**Experience:** 7+ years
**Skills:** Distributed Systems, Test Automation, Chaos Engineering, Fault Injection, Load Testing, Performance Testing, Traffic Replay, Observability, Tracing, Workload Generation, Multi-Tenant Testing, Release Pipelines
**Posted:** 2026-08-26

> Leads development of shared test systems and tooling that reproduce production workloads, validate failure modes, and expand system-level coverage. Requires strong software engineering fundamentals, technical leadership, and experience with production-quality platforms or distributed systems.

## Job Description

## Responsibilities
- Build production-like test environments and tooling for test cells that resemble production topology, configuration, and behavior.
- Develop workload-generation capabilities for multi-tenant, stress, load, and traffic-replay testing.
- Create repeatable failure-mode testing tools for latency, dependency faults, resource exhaustion, degraded-state recovery, and regional failures.
- Identify high-impact system-level testing gaps and implement repeatable continuous or release validation coverage.
- Build shared libraries, test harnesses, frameworks, and paved paths for system and integration testing.
- Mentor engineers and influence testing strategy through design documents, code reviews, and cross-team collaboration.
- Partner with Cloud/Infrastructure, Reliability, Release Engineering, and product teams to incorporate incident learnings into pre-production coverage and release pipelines.

## Requirements
- Strong software engineering fundamentals and experience building and operating production-quality platforms, infrastructure, developer tooling, or distributed systems.
- Demonstrated technical leadership of ambiguous, cross-team initiatives, including alignment and delivery of durable shared systems.
- Experience designing for observability, debuggability, and operational readiness, including metrics, logs, tracing, safe rollouts, and failure analysis.
- Ability to collaborate across organizational boundaries.
- Strong written communication skills for design documents, testing strategy proposals, and scalable documentation.

## Nice-to-Haves
- Chaos engineering, fault injection, workload replay or shadowing, performance testing, or multi-tenant test strategy.
- Experience building broadly used internal platforms, self-service environments, paved roads, or shared frameworks.
- Familiarity with engineering and system outcome metrics, including signal quality, coverage gaps, incident-to-coverage time, and adoption.

## Compensation and Benefits
- Estimated pay range: **$212,000–$278,250** depending on experience and location.
- Eligible to participate in Temporal's equity plan.
- Full-time benefits include unlimited paid time off, holidays, medical, dental and vision coverage, disability and life insurance, a 401(k) plan, learning and development support, lifestyle spending, home office setup, professional memberships, work-from-home meals, internet stipend, and wellness resources.
- Occasional travel may be required for company events, team offsites, and other in-person collaboration.

## Similar jobs

- [Wireless Firmware Validation Engineer](https://hotfix.jobs/jobs/7d7a5245-91fc-48d1-beec-44ef9c0c5c9d) - Zoox - Foster City, CA - $206k – $240k/yr
- [Quality Engineer, Rack Infrastructure & Site Operations](https://hotfix.jobs/jobs/f235884c-5128-4963-93e3-763f30b7eaaf) - OpenAI - San Francisco, CA - $205k – $344k/yr
- [Staff Engineer](https://hotfix.jobs/jobs/c2cf523f-82e0-4741-accd-091fd64ea301) - Illumio - Sunnyvale, CA - $194k – $233k/yr
- [Staff Performance Quality Engineer](https://hotfix.jobs/jobs/3874dd15-8b31-42b8-aa18-89d6b65d811e) - Domino - Remote - $185k – $210k/yr
- [Staff Quality Assurance Engineer](https://hotfix.jobs/jobs/cb8d431e-422f-4d87-ba4c-ca1c33bf489e) - Second Nature - Remote - $180k – $200k/yr

**Apply:** https://hotfix.jobs/jobs/66bfa758-b4c1-4fcf-8ca6-fb26f63022c4
**Canonical:** https://hotfix.jobs/jobs/66bfa758-b4c1-4fcf-8ca6-fb26f63022c4