Staff+ Software Engineer, Data Infrastructure
Build and scale secure data infrastructure at Anthropic, including access control systems, financial data pipelines, cloud storage reliability, and analytics tooling. Requires 10+ years building data/distributed systems, 3+ years leading complex projects, and deep experience with cloud/IaC or programming languages.
About the job
Responsibilities
Within Data Infra, you may be matched to critical business areas including:
Data Governance & Access Control: Design and implement robust access control systems ensuring only authorized users can access sensitive data. Build infrastructure for permission management, audit logging, and compliance requirements. Work on IAM policies, ACLs, and security controls that scale across thousands of users and systems.
Financial Data Infrastructure: Build and maintain data pipelines and warehouses powering business-critical reporting. Ensure data integrity, accuracy, and availability for complex financial systems, including third party revenue ingestion pipelines; manage the external relationships as needed to drive upstream dependencies. Own the reliability of systems processing revenue, usage, and business metrics.
Cloud Storage & Reliability: Architect disaster recovery, backup, and replication systems for petabyte-scale data. Ensure high availability and durability of data stored in cloud object storage (GCS, S3). Build systems that protect against data loss and enable rapid recovery.
Data Platform & Tooling: Scale data processing infrastructure using technologies like BigQuery, BigTable, Airflow, dbt, and Spark. Optimize query performance, manage costs, and enable self-service analytics across the organization.
Requirements
- 10+ years (not including internships or co-ops) of experience in a Software Engineer role, building data infrastructure, storage systems, or related distributed systems
- 3+ years (not including internships or co-ops) of experience leading large scale, complex projects or teams as an engineer or tech lead
- Can set technical direction for a team, not just execute within it
- Deep experience with at least one of:
- Strong proficiency in programming languages like Python, Go, Java, or similar
- Experience with infrastructure-as-code (Terraform, Pulumi) and cloud platforms (GCP, AWS)
- Can navigate complex technical tradeoffs between performance, cost, security, and maintainability
- Excellent collaboration skills - you work well with both technical and non-technical stakeholders
- Bachelor’s degree or an equivalent combination of education, training, and/or experience in a field relevant to the role
Nice-to-Haves
- Experience with security and compliance requirements (ITGC, GDPR, financial controls)
- Background in data warehousing, ETL/ELT pipelines, or analytics infrastructure
- Experience with Kubernetes, containerization, and cloud-native architectures
- Track record of improving data reliability, availability, or cost efficiency at scale
- Knowledge of column-oriented databases, OLAP systems, or big data processing frameworks
- Experience working in fintech, financial services, or highly regulated environments
- Security engineering background with focus on data protection and access controls
Technologies
- Data: BigQuery, BigTable, Airflow, Cloud Composer, dbt, Spark, Segment, Fivetran
- Storage: GCS, S3
- Infrastructure: Terraform, Kubernetes, GCP, AWS
- Languages: Python, Go, SQL
Skills
Python, Go, Java, Terraform, Pulumi, GCP, AWS, Kubernetes, BigQuery, Spark, Airflow, dbt, SQL
Similar jobs
Data Engineering jobs
Anthropic
San Francisco, CABuild and operate the data foundations behind AI safeguards, including production pipelines, data stores, governance controls, and internal tooling. The role requires strong Python and SQL skills, production data-platform experience, and expertise in reliable, privacy-conscious systems across multiple cloud environments.
The Voleon Group
New York, NYLeads the design and delivery of scalable data infrastructure, services, and developer tooling while shaping technical direction and mentoring engineers. Requires 8+ years of software engineering experience, strong systems design expertise, and proficiency in a modern programming language.
Vercel
San Francisco, CAStaff Data Platform Engineer leading the architecture and development of financial data infrastructure for revenue reporting, billing, forecasting, and compliance. Requires 8+ years of data engineering or architecture experience, strong streaming and warehouse expertise, and the ability to mentor engineers and partner with Finance and Audit leaders.
Vanta
Remote
Leads the re-platforming of Vanta’s compliance data layer from MongoDB to schema-aware PostgreSQL across high-throughput Kafka and S3 pipelines. The role requires staff-level distributed systems expertise, migration leadership, and strong experience with relational and document data modeling.
Anyscale
San Francisco, CA
Staff Software Engineer responsible for designing and scaling Ray Data’s distributed data-processing infrastructure for large-scale AI training and inference. Requires 6+ years of production software and architectural ownership experience, plus deep distributed-systems expertise and strong Python skills.