Senior/Staff Software Engineer, Data Platform
Build scalable data platforms for autonomous driving ML systems, including batch/streaming pipelines, storage, dashboards, and monitoring. Requires 4+ years experience in large-scale data systems, Python/C++, and engineering leadership.
About the job
Responsibilities
- Design and develop unified, introspectable, large-scale batch and streaming data processing systems that can ingest and process data across a wide range of use cases relevant to evaluation.
- Create and implement a storage system capable of accommodating both the large volume and diverse range of evaluation and performance metrics.
- Construct intuitive dashboards and reports to present evaluation results, facilitating straightforward comparisons that highlight both improvements and regressions.
- Design and develop comprehensive end-to-end data pipelines that streamline the flow from data ingestion to final consumption.
- Develop and maintain continuous testing and monitoring systems to guarantee the integrity and resilience of our data and associated data pipelines.
Requirements
- B.Sc or M.Sc. plus 4+ years of relevant work experience
- Strong proficiency in Python, C++, or similar languages
- Domain experience: Experience working with large scale data and building scalable & reliable systems / data pipelines; ability to understand and design complex systems
- Engineering leadership: Experience setting team or project product and technical vision, timelines and prioritization; formally or informally being a Tech Lead, mentoring and support junior engineers
- Technical excellence: Ability and willingness to deep dive into implementation, driving technical standards and best practices across broader software organization
- Bachelor's degree in Computer Science, Electrical Engineering, or a closely related field
Nice-to-Haves
- Strong knowledge of GCP, GCS, BigQuery, or PostgreSQL
- Knowledge of data engineering, and its tooling and best practices
- Knowledge of batch and streaming data processing, warehousing, and analytics solutions
- Experience working with large scale distributed data systems
- Experience with system & framework design
- Experience with data workflow orchestration platforms
Compensation
- Base pay range: $193,930 - $352,290
- Eligible for annual performance bonus, equity, and competitive benefits package
Skills
Python, C++, GCP, BigQuery, Postgres, Gcs, Data Pipelines, Batch Processing, Streaming Processing, Data Orchestration
Similar jobs
Data Engineering jobs
Databricks
New York, NY
Leads large-scale advertising data ingestion, measurement, and agentic workflow systems, combining deep ad-tech expertise with production LLM experience. Requires 10+ years of engineering experience and technical and people leadership in complex enterprise environments.
Rippling
San Francisco, CABuild and scale data ingestion platforms, pipelines, APIs, and processing products that move billions of rows across a multi-tenant system. The role requires 8+ years of software development experience and strong expertise in large-scale application architecture.
Webflow
United States
Staff Data Engineer leading data platform initiatives across batch, streaming, real-time pipelines, data lake infrastructure, governance, and privacy. Requires 5+ years of data engineering experience, strong Spark and distributed processing expertise, and the ability to lead complex production systems.
Airbnb
United States
Staff Software Engineer leading design and development of large-scale batch and real-time data pipelines and ML infrastructure to power GenAI/LLM products and features for Airbnb's Messaging, Notifications, and Connectivity organization. Requires 9+ years experience building production ML systems and cross-functional collaboration.
Coinbase
United States
This staff-level data engineer will architect and operate low-latency market data infrastructure, including feed handling, normalization, distribution, and exchange connectivity. The role requires at least five years of backend engineering experience and strong Java or C++ expertise with high-throughput messaging and market data protocols.