Staff+ Software Engineer, Capacity Engineering
Build and operate production data pipelines, observability tools, and planning systems to maximize utilization, efficiency, and attribution of Anthropic's large-scale multi-cloud accelerator and CPU fleet. Requires strong Python/SQL, cloud operations, and Kubernetes experience in a high-ambiguity environment.
About the job
Key Responsibilities
- Build the planning and allocation stack for capacity allocation, including cross-region and cross-provider placement, guardrails, queueing, and occupancy KPIs.
- Drive efficiency programs such as stranding and rightsizing, unused capacity recovery, and job-level utilization across training, inference, and eval workloads. Establish per-config baselines and collaborate with system-owning teams to improve utilization.
- Own attribution and forecasting: reconcile billing across multiple providers against telemetry and internal systems, attribute spend to workloads, and convert demand signals and research roadmaps into compute plans.
- Build the data platform: pipelines ingesting occupancy, utilization, and cost data from a diversifying fleet into BigQuery, ensuring completeness, latency SLOs, and gap detection.
- Operate Kubernetes-native systems at scale, including collection agents, workload labeling, and taint/reservation/scheduling behaviors.
- Treat outputs as products: gather requirements, define schema contracts, design for diverse consumers (research engineers to CFO), and maintain on-call and SLOs.
What You Bring
- Strong track record building and operating production systems (hands-on engineering role with DevOps flavor).
- Production-quality Python and SQL (pipeline code in Python; BigQuery SQL including table-valued functions and views; idiomatic, tested, maintainable).
- Deep experience with at least one major cloud provider (AWS, Google Cloud, or Azure) and its operations.
- Experience with observability tooling including Prometheus, PromQL, and Grafana (writing recording rules and building relied-upon monitoring).
- Ability to gather own requirements and work across organizational boundaries in ambiguous environments with limited direction.
Preferred Qualifications
- Experience with capacity planning, resource management, or cost attribution at a hyperscaler or in large-scale ML environments (product engineering and developer experience counts).
- Scheduling/packing efficiency or profiling-driven optimization of large distributed workloads.
- Multi-cloud data ingestion, especially normalizing billing exports, reservation APIs, commitments, and vendor telemetry.
- Total cost of ownership and forecasting, including decomposing infrastructure growth drivers.
- Accelerator infrastructure familiarity (GPU metrics like DCGM, TPU utilization, Trainium metrics, or ML training/inference at hardware level).
- Building internal data products with self-service access, schema contracts, APIs, documentation, and discoverability.
- Storage efficiency, retention, and lifecycle at exabyte scale.
Compensation
Annual Salary: $320000–$485000 USD
Minimum education: Bachelor’s degree or equivalent.
Skills
Python, SQL, BigQuery, Kubernetes, Prometheus, Promql, Grafana, AWS, GCP, Azure, Data Pipelines
Similar jobs
Data Engineering jobs
Anthropic
San Francisco, CABuild and operate the data foundations behind AI safeguards, including production pipelines, data stores, governance controls, and internal tooling. The role requires strong Python and SQL skills, production data-platform experience, and expertise in reliable, privacy-conscious systems across multiple cloud environments.
The Voleon Group
New York, NYLeads the design and delivery of scalable data infrastructure, services, and developer tooling while shaping technical direction and mentoring engineers. Requires 8+ years of software engineering experience, strong systems design expertise, and proficiency in a modern programming language.
Vercel
San Francisco, CAStaff Data Platform Engineer leading the architecture and development of financial data infrastructure for revenue reporting, billing, forecasting, and compliance. Requires 8+ years of data engineering or architecture experience, strong streaming and warehouse expertise, and the ability to mentor engineers and partner with Finance and Audit leaders.
Vanta
Remote
Leads the re-platforming of Vanta’s compliance data layer from MongoDB to schema-aware PostgreSQL across high-throughput Kafka and S3 pipelines. The role requires staff-level distributed systems expertise, migration leadership, and strong experience with relational and document data modeling.
Anyscale
San Francisco, CA
Staff Software Engineer responsible for designing and scaling Ray Data’s distributed data-processing infrastructure for large-scale AI training and inference. Requires 6+ years of production software and architectural ownership experience, plus deep distributed-systems expertise and strong Python skills.