Data & ML Pipeline Software Engineer
Builds large-scale data processing pipelines and ML infrastructure to automate data curation, model training, and iteration for autonomous vehicles using real-world and simulation data. Requires 3-5 years experience in data/ML infra, Python, and frameworks like Spark/Airflow/Kafka.
About the job
Responsibilities
- Build and maintain large-scale data processing pipelines (ETL) for ingesting and curating driving datasets.
- Design and implement systems that automate data selection, labeling, training, and testing loops.
- Collaborate with modeling teams to improve training efficiency and model performance across iterations.
- Develop the core infrastructure that closes the loop between real-world test results and new model deployments.
- Use engineering expertise to help vehicles learn from data at scale, improving safety and performance.
- Mentor junior engineers and contribute to defining best practices for data-centric development.
Requirements
- Bachelor's or higher degree in Engineering such as Computer Science, Electrical Engineering, Software Engineering.
- 3–5 years of experience in software or data infrastructure engineering.
- Expertise in building and scaling data pipelines, distributed systems, or ML infrastructure.
- Proficiency in Python and strong knowledge of data frameworks (Spark, Airflow, Kafka, etc.).
- Experience working with large-scale datasets and understanding data-driven development cycles.
- Familiarity with machine learning workflows or model training/deployment, especially automation of those processes.
- Strong systems thinking and ability to work across multiple parts of the stack (data, infra, and ML).
- Interest in seeing the direct impact of infrastructure work on vehicle performance.
Nice to Have
- Experience with automotive (AV) or robotics systems.
- Previous work on ML platforms for large-scale products (e.g., Ads, Recommendation, or Autonomy pipelines).
- Experience with highly automated ML training workflows.
- Prior contributions to systems that connect data-driven model iteration loops ("data flywheel").
- Ability to move fast, learn quickly, and mentor others while growing with the team.
Compensation
- Base salary range: $125,000 - $222,000 USD annually.
- Equity, comprehensive health/dental/vision/life/disability insurance, 401k with employer match, learning/wellness stipends, paid time off.
Skills
Python, Spark, Airflow, Kafka, Distributed Systems, ETL, Machine Learning, Data Pipelines, ML Infrastructure
Similar jobs
Data Engineering jobs
San Francisco, CA
Build and scale AWS-based data infrastructure, pipelines, knowledge graphs, and APIs for a CTV performance advertising platform. The role requires production data engineering experience with Spark, Scala, AWS, SQL, and large-scale services, plus a bachelor's degree.
Bevi
Boston, MA
Build and maintain dbt models, Snowflake semantic layers, and ingestion pipelines across business functions while improving data quality and resilience. The role requires 4–6 years of analytics or data engineering experience, strong dbt and SQL expertise, and a quantitative bachelor's degree.
Stuut
San Francisco, CABuild and own Stuut’s foundational data platform, including ingestion pipelines, canonical models, semantic layers, and observability. The role requires 3+ years of production data pipeline experience with Python, SQL, cloud warehouses, and ETL/ELT tooling.
Ontic
United States
Build scalable analytics engineering infrastructure, SaaS data models, and AI-enabled workflows that support enterprise decision-making. The role requires 3–6 years of hands-on analytics or data engineering experience, strong SQL and modern data modeling expertise, and cloud data warehouse experience.
Underdog Fantasy
United States
Build and operate the data platform supporting automated regulatory reporting for a prediction markets business. The role combines SQL and dbt development, end-to-end data investigation, automated validation, and cross-functional ownership under strict deadlines.