Software Engineer, Data Platform
Builds scalable data infrastructure for autonomous driving ML systems, processing large-scale batch/streaming data from logs and simulations. Requires Python/C++ proficiency, data pipeline experience, and bachelor's in CS/EE; 1+ years experience.
About the job
Responsibilities
- Design and develop unified, introspectable, large-scale batch and streaming data processing systems that ingest and process data across evaluation use cases.
- Create and implement storage systems for large volume and diverse evaluation and performance metrics.
- Construct intuitive dashboards and reports for evaluation results, enabling comparisons of improvements and regressions.
- Design and develop end-to-end data pipelines from ingestion to consumption.
- Develop and maintain continuous testing and monitoring systems for data and pipeline integrity.
Requirements
- B.Sc or M.Sc. plus 1+ years of relevant work experience.
- Strong proficiency in Python, C++, or similar languages.
- Experience with large scale data and building scalable & reliable systems/data pipelines.
- Ability to understand and design complex systems.
- Bachelor's degree in Computer Science, Electrical Engineering, or related field.
Nice-to-Haves
- Strong knowledge of GCP, GCS, BigQuery, or PostgreSQL.
- Knowledge of data engineering, tooling, and best practices.
- Knowledge of batch and streaming data processing, warehousing, and analytics.
- Experience with large scale distributed data systems.
- Experience with system & framework design.
- Experience with data workflow orchestration platforms.
Compensation
- Base pay range: $160,360 - $240,540 (depending on experience, qualifications, etc.).
- Eligible for annual performance bonus, equity, and competitive benefits.
Skills
Python, C++, GCP, BigQuery, Postgres, Data Pipelines, Apache Beam, Airflow, Kubernetes, Distributed Systems
Similar jobs
Data Engineering jobs
Garner Health
New York, NY
Build and optimize scalable data pipelines, reusable datasets, and federated data quality systems for healthcare analytics. The role requires at least 2 years of data or software engineering experience and strong Python, SQL, AWS, orchestration, database, and warehouse expertise.
Astera
Washington, DC
Manages the full lifecycle of scientific research data, including governance, metadata, repositories, open-science publishing, and AI/ML compatibility. The role also builds partnerships with federal research organizations and requires a bachelor’s degree plus 5–7+ years of relevant experience.
Vercel
San Francisco, CABuild and operate the streaming, storage, query, and self-service infrastructure underlying the company’s data platform. The role suits an early-career engineer with 1–3 years of experience, a computer science bachelor’s degree, programming skills, and interest in distributed systems.
Baselayer
San Francisco, CA
Build and operate production data pipelines and transformation layers that turn heterogeneous business, identity, and fraud data into reliable inputs for entity resolution, scoring, and customer APIs. The role requires at least one year of data engineering experience with Python, SQL, cloud platforms, and modern pipeline tooling.
TheGuarantors
New York, NY
Build and maintain reliable data pipelines, warehouses, and lightweight data applications supporting analytics, operational models, and AI/ML workflows. The role requires SQL, Python, ETL, and data warehousing knowledge, with 1+ year of relevant experience preferred.