Software Engineer - Data Platform
Builds and operates petabyte-scale data platform infrastructure using Kafka, Spark, Flink, and Trino to power real-time ML pipelines and analytics. Requires expertise in distributed systems, stream processing, and systems languages like Rust, Go, or Scala.
About the Team
The Data Platform team builds and operates infrastructure for large-scale data transport and processing, managing Apache Kafka, HDFS, Spark, Flink, and Trino for real-time ML pipelines, feed ranking, experimentation, analytics, and observability at petabyte scale.
About the Role
Design, build, and operate distributed systems powering data movement and compute, processing trillions of events daily for scalability, performance, and reliability in product and ML workloads.
What You Will Do
- Design and implement high-throughput, low-latency data ingestion and transport systems.
- Scale and optimize multi-tenant Kafka infrastructure supporting real-time workloads.
- Extend and tune Spark, Flink, and Trino for demanding production pipelines.
- Build interfaces, APIs, and pipelines enabling teams to query, process, and move data at petabyte scale.
- Debug and optimize distributed systems, with a focus on reliability and performance under load.
- Collaborate with ML, product, and infrastructure teams to unblock critical data workflows.
Ideal Candidate
- Proven expertise in distributed systems, stream processing, or large-scale data platforms.
- Proficiency in Rust, Go, Scala or similar systems languages.
- Hands-on experience with Kafka, Flink, Spark, Trino, or Hadoop in production.
- Strong debugging, profiling, and performance optimization skills.
- Track record of shipping and maintaining critical infrastructure.
- Comfortable working in fast-moving, high-stakes environments with minimal guardrails.
Compensation and Benefits
Annual Salary Range: $180,000 - $440,000 USD
Base salary plus equity, comprehensive medical, vision, dental, 401(k), disability insurance, life insurance, and perks.
Staff Data Platform Engineer
Staff Data Platform Engineer building and leading AWS-native data platform architecture, orchestration, governance, and AI-readiness for analytics and ML workloads. Requires 8-10+ years experience with AWS data systems and strong technical leadership.
Manager, Data Engineering
Lead and mentor a team of data engineers building scalable data pipelines and platform infrastructure. Hands-on coding, operational excellence, and cross-functional collaboration with analytics, data science, and business teams.
Senior Data Engineer, People Analytics
Build and maintain data pipelines, tables, and AI-ready data foundations from HR systems to power People Analytics reporting, dashboards, and LLM tools. Requires 5+ years of data engineering experience with strong SQL, Python, Airflow, and data governance skills.