Senior Software Engineer, Data Transformation
Build and operate high-throughput streaming systems and database internals for Snowflake’s cloud-scale data platform. The role requires 5+ years of distributed-systems or related experience, strong programming skills, and expertise in correctness, fault tolerance, and state management.
About the job
Responsibilities
- Design and implement high-throughput stream-processing systems with strong correctness guarantees, including exactly-once semantics, out-of-order event handling, and low-latency execution.
- Work on database internals, including query execution, operator design, storage engine components, and transaction processing at cloud scale.
- Own features and systems end to end: design, implementation, testing, deployment, and production observability.
- Drive architectural decisions shaping the long-term evolution of streaming and core platform layers.
- Mentor engineers through design reviews, code reviews, and technical guidance.
- Partner across product, infrastructure, and research teams to deliver foundational capabilities.
Requirements
- 5+ years of software engineering experience in distributed systems, query engines, streaming infrastructure, or database internals, or equivalent experience including research.
- Bachelor's, master's, or doctoral degree in Computer Science or a related field; graduate research in streaming systems, query processing, or database engines is a strong differentiator.
- Deep expertise in distributed systems, including consistency, fault tolerance, replication, and state management.
- Hands-on experience building, operating, or extending stream-processing systems in industry or academia.
- Proficiency in multiple languages such as Java, Scala, C++, and Python, with a record of delivering production systems at scale.
- Proficiency with AI-native software engineering and agentic development workflows.
- Ability to lead complex technical projects independently with minimal direction.
Nice-to-haves
- Experience with stream-processing engine internals, such as state backends, checkpointing, runtimes, or operator development, in Apache Flink, Spark Structured Streaming, or similar systems.
- Experience with SQL engine internals, including query optimization, execution-plan design, storage formats, or transaction layers.
- Published research or thesis work in streaming, real-time query processing, or distributed computation.
- Contributions to open-source data infrastructure or database systems.
- Familiarity with Spec Driven Development (SDD) and Test Driven Development (TDD).
- Familiarity with formal verification.
Skills
Distributed Systems, Stream Processing, Database Internals, Query Execution, Java, Scala, C++, Python, Apache Flink, Spark Structured Streaming, Sql Optimization, Replication, Fault Tolerance, Test Driven Development, Formal Verification
Similar jobs
Data Engineering jobs
Addepar
United States
Staff Software Engineer building scalable frameworks for high-performance financial data ingestion/distribution and AI-native products. Requires 7+ years experience with distributed systems, microservices, and data architectures; partners with product teams to drive technical direction.
Alpaca
Remote
Build and operate scalable lakehouse infrastructure, streaming and CDC pipelines, query systems, and self-serve BI capabilities. Requires 5+ years of data engineering experience, strong Kubernetes and infrastructure-as-code expertise, and hands-on experience with distributed data platforms.
NinjaTrader
Chicago, IL
Leads database architecture, performance, reliability, and developer-tooling initiatives for high-volume trading applications. Requires 8+ years of software engineering experience, expert MySQL skills, backend development expertise, and strong knowledge of distributed systems and database operations.
Flaglerhealth
Vancouver, Canada
Senior Data Engineer responsible for building and operating reliable clinical and claims data pipelines, CDC systems, quality controls, and de-identified exports. The role requires 5+ years of production pipeline experience plus strong SQL, Python, Spark, and data-governance skills.
Deepgram
United States
Own the company’s metric governance program by defining canonical metrics, enforcing them in semantic and catalog systems, improving data quality, and validating AI-agent outputs. Requires 5+ years in analytics or analytics engineering, strong SQL, production semantic-layer ownership, and experience with AI evaluation and data governance.