Principal Engineer, Big Data Platform
Leads the strategy, architecture, and scaling of Pinterest’s big data and AI infrastructure across petabyte-scale workloads. Requires principal-level technical leadership, extensive Kubernetes or big data platform experience, and proficiency in modern data and cloud technologies.
About the job
Responsibilities
- Lead the strategy and technical direction of Pinterest’s data infrastructure for big data and AI applications.
- Build and scale infrastructure frameworks for petabyte-scale datasets, including compute engines, job management, resource management, scheduling, and remote shuffling.
- Work with internal customers on critical business use cases that rely on big data.
- Provide company-wide technical guidance on processing and storing data reliably, efficiently, and at scale.
- Contribute to the team’s technical vision and long-term roadmap.
Requirements
- 12+ years of industry experience with a proven track record of technical excellence.
- 8+ years building and supporting large-scale Kubernetes or big data platforms.
- Deep knowledge of big data and machine learning technologies.
- Proficiency in one or more programming languages.
- Experience with Kubernetes and AWS technologies.
- Exceptional collaboration skills, including navigating ambiguity, making tradeoffs, and aligning stakeholders.
- Bachelor’s degree in Computer Science, a related technical field, or equivalent experience.
Compensation
- Base salary range: $242,634–$499,541 USD.
- Eligible for equity.
Skills
Apache Flink, Spark, Kubernetes, Presto, Ray, PyTorch, TensorFlow, Java, Go, Scala, Python, AWS
Similar jobs
Data Engineering jobs
ZoomInfo
United States
Architects and builds ZoomInfo’s distributed data-platform infrastructure, including federated GraphQL access, real-time pipelines, indexing, and observability. The role requires 10+ years of software engineering experience, cloud-native expertise, and strong distributed-systems design skills.
Orum
Austin, TX
Own the architecture, reliability, and evolution of a modern data platform spanning data engineering and analytics/BI. The role requires 8+ years of experience plus deep expertise in SQL, dbt, ClickHouse, BigQuery, Looker, streaming systems, and distributed data platforms.
Octus
United States
Leads the strategy, architecture, and hands-on development of scalable AWS data ingestion and transformation platforms. Requires expert Python and SQL skills, Terraform and cloud-native pipeline experience, and 8+ years in data engineering or backend development, with technical leadership responsibilities.
Anyscale
San Francisco, CA
Staff Software Engineer responsible for designing and scaling Ray Data’s distributed data-processing infrastructure for large-scale AI training and inference. Requires 6+ years of production software and architectural ownership experience, plus deep distributed-systems expertise and strong Python skills.
Haus
San Francisco, CAStaff-level engineer leading backend services and data-platform architecture, including large-scale ingestion, distributed systems, and trustworthy BigQuery/dbt warehouse models. Requires 10+ years of software engineering experience, expert Python, deep SQL/dbt expertise, and strong technical leadership.