Software Engineer - Kafka
Builds large-scale data platforms for AI and autonomy, processing petabytes of sensor data using distributed systems like Kafka and Spark. Requires 2+ years backend experience and bachelor's in CS.
About the job
Responsibilities
- Design and build large-scale data platforms to support AI research and autonomy stack development, handling petabytes of multimodal sensor data from real-world driving scenarios.
- Work on data curation and tagging platforms that enable efficient dataset discovery, labeling workflows, and quality assessment across diverse driving conditions.
- Build high-performance data processing systems using modern distributed computing frameworks to transform raw sensor data into training-ready formats.
Requirements
- Bachelor's degree in Computer Science, Software Engineering, or equivalent.
- 2+ years of professional experience.
- Strong backend engineering experience.
- Problem-solving skills and experience working with cross-functional teams in fast-paced environments.
Nice to Have
- Hands-on experience with modern data stack technologies including Apache Spark, Hudi, Trino, Kafka, or similar distributed data processing frameworks.
- Knowledge of data lake architectures, streaming systems, and workflow orchestration platforms like Flyte.
Compensation
- Base salary range: $153,000 - $222,000 USD annually.
- Equity, comprehensive health/dental/vision/life/disability insurance, 401k with employer match, learning/wellness stipends, paid time off.
Skills
Apache Kafka, Spark, Apache Hudi, Trino, Flyte, Kubernetes, Python, Go, Java
Similar jobs
Backend Engineering jobs
Lyft
Seattle, WA
Develop and operate scalable backend systems and security detection capabilities, collaborating cross-functionally from project inception through launch. Requires a master’s degree and at least two years of software development experience, including object-oriented programming and large-scale distributed systems.
Stripe
New York, NY
Build backend systems, APIs, and user experiences for Stripe’s merchant products and infrastructure. The role requires a bachelor’s degree and at least two years of software development experience across multiple languages, cloud services, and data technologies.
Nuro
Mountain View, CA
Build real-time video-streaming pipelines, simulation frameworks, and cloud communication systems for a vehicle teleoperations platform. The role requires C/C++ or Go, networking expertise, and a Computer Science degree, with 2+ years of relevant experience for BS/MS candidates.
Bubble
Remote
Build and improve high-scale backend infrastructure, workflow systems, data tiers, and reliability tooling supporting billions of requests. The role requires at least two years of software engineering experience, strong coding and problem-solving skills, distributed-systems familiarity, and participation in on-call operations.
Illumio
Sunnyvale, CA
Develop and operate Go-based containerized microservices that process cloud telemetry and real-time events for a multi-tenant security platform. The role requires 2+ years building distributed systems, cloud API experience, and strong SQL, networking, and Linux skills.