Principal Software Engineer - Data Engineering & Streaming Primitives
Principal-level engineer to define and lead Snowflake's core data engineering and streaming primitives (Streams, Tasks, Dynamic Tables) at cloud scale. Requires 15+ years building large-scale distributed data systems and deep expertise in stream processing or data transformation.
About the job
What You'll Do
- Define and drive the technical direction for Snowflake's core data engineering and streaming transformation primitives, spanning Streams, Tasks, Dynamic Tables, and adjacent pipeline constructs.
- Identify and lead multi-quarter technical investments — performance, scalability, correctness, and reliability — translating ambiguous problem spaces into concrete engineering plans with measurable outcomes.
- Partner with product, research, and peer engineering teams to co-design primitives that compose cleanly across the data engineering stack.
- Operate as a force multiplier: run architectural reviews, set the technical bar for design documents, and help engineers grow through high-quality feedback and sponsorship.
- Work directly with customers and field teams to understand real-world usage patterns; use that signal to prioritize what matters next.
- Contribute to Snowflake's technical reputation — through internal design influence, external talks, or research publications in the data engineering space.
What We're Looking For
- 15+ years of experience designing, building, and operating large-scale distributed data systems.
- Deep expertise in at least one core area: stream processing, declarative query execution, pipeline orchestration, or data transformation at scale.
- Strong computer science fundamentals — distributed systems, algorithms, fault tolerance, and consistency models.
- Proficiency in C++ or Java; comfort with systems-level reasoning (latency, throughput, resource efficiency at cloud scale).
- Demonstrated ability to lead cross-team technical initiatives from blank-page architecture through production at petabyte scale across thousands of concurrent workloads.
- Strong written and verbal communication skills; ability to represent complex technical trade-offs clearly to engineering, product, and leadership audiences.
Nice to Have
- Experience with a major analytical DBMS (Snowflake, BigQuery, Redshift, Databricks, Teradata).
- Hands-on background in streaming or event-driven systems (Flink, Kafka, Spark Structured Streaming).
- Familiarity with the broader data engineering ecosystem: dbt, Airflow, Fivetran, Iceberg, Delta Lake.
- Experience with CDC, change propagation, or incremental computation patterns.
- Advanced degree (MS or PhD) in Computer Science, with emphasis on database or distributed systems.
Skills
C++, Java, Distributed Systems, Stream Processing, Data Transformation, Fault Tolerance, Consistency Models, Pipeline Orchestration, Declarative Query Execution, Petabyte Scale Systems
Similar jobs
Data Engineering jobsLeads the strategy, architecture, and scaling of Pinterest’s big data and AI infrastructure across petabyte-scale workloads. Requires principal-level technical leadership, extensive Kubernetes or big data platform experience, and proficiency in modern data and cloud technologies.
Architects and builds ZoomInfo’s distributed data-platform infrastructure, including federated GraphQL access, real-time pipelines, indexing, and observability. The role requires 10+ years of software engineering experience, cloud-native expertise, and strong distributed-systems design skills.
Own the architecture, reliability, and evolution of a modern data platform spanning data engineering and analytics/BI. The role requires 8+ years of experience plus deep expertise in SQL, dbt, ClickHouse, BigQuery, Looker, streaming systems, and distributed data platforms.
Leads the strategy, architecture, and hands-on development of scalable AWS data ingestion and transformation platforms. Requires expert Python and SQL skills, Terraform and cloud-native pipeline experience, and 8+ years in data engineering or backend development, with technical leadership responsibilities.
Staff Data Platform Engineer leading the architecture and development of financial data infrastructure for revenue reporting, billing, forecasting, and compliance. Requires 8+ years of data engineering or architecture experience, strong streaming and warehouse expertise, and the ability to mentor engineers and partner with Finance and Audit leaders.