Principal Software Engineer - Data Lakes
Leads architecture and development of scalable, high-performance managed data lake systems for customers. The role requires deep relational data management and performance-engineering experience, strong Java and C++ skills, public-cloud expertise, and technical leadership.
About the job
Responsibilities
- Partner closely with product teams to understand requirements and design new capabilities for customers.
- Design, develop, implement, and operate highly reliable, large-scale data lake systems with a dedicated data lake engineering team.
- Contribute to open-source projects such as DuckDB and Polaris.
- Embrace open-source standards and toolsets.
- Analyze fault-tolerance, high-availability, performance, and scale challenges, and solve them.
- Ensure operational excellence and meet customer commitments for security, reliability, availability, and performance.
- Set technical direction and influence cross-functional teams.
Requirements
- 6+ years of hands-on or research experience with high-performance relational data management systems.
- Deep understanding of infrastructure and software optimization, performance engineering, latency, and availability improvements.
- Proven track record of leading and delivering large, complex projects.
- Strong development skills in Java and C++.
- Solid experience with public clouds, including AWS, Azure, and Google Cloud.
- Demonstrated knowledge of columnar storage formats.
- Growth mindset and enthusiasm for innovative solutions.
- Excellent teamwork and commitment to helping colleagues improve.
Nice-to-haves
- Master's or PhD in Computer Science, ideally focused on database management and/or storage engines.
Compensation and Benefits
- 100% employer-paid medical insurance.
- Generous paid time off, paid sick time, inclusive parental leave, holidays, a year-end Global Week of Rest, and volunteer days off.
- RSU stock grants.
- Professional development and training opportunities.
- Virtual happy hours, free food, and team-building activities.
- Monthly cell phone stipend.
- Access to a mental health support platform offering therapy, coaching, and mindfulness resources for employees and dependents.
- Benefits may vary by country and worker type.
Skills
Java, C++, Postgres, Temporal, gRPC, AWS, GCP, Azure, Kubernetes, Grafana, Apache Iceberg, Polaris, Delta Lake, Parquet, Duckdb
Similar jobs
Backend Engineering jobs
Payabli
Remote
Founding senior engineer responsible for creating and operating Payabli’s Payments Core infrastructure, including ledgers, exactly-once money movement, multi-rail orchestration, settlement, and reconciliation. Requires 12+ years of production payments or financial-systems experience at major scale.
Anaconda
United StatesStaff Software Engineer leading the production toolchain for compiling and distributing complex Python packages across Linux, macOS, and Windows. Requires 10+ years of software development experience and deep C/C++ expertise in compilation, linking, architecture, and technical strategy.
Vanta
Remote
Leads architecture and technical strategy for Vanta’s government compliance product, building scalable backend systems and new workflows in a 0-to-1 environment. Requires 10+ years of backend experience, strong systems thinking, cross-functional influence, and a track record of mentoring engineers.
Databricks
Amsterdam, NetherlandsBuild scalable backend infrastructure and products for the Databricks platform, including distributed services, cloud storage systems, data pipelines, and pricing infrastructure. Requires a bachelor's degree and 7+ years of production software engineering experience.
Databricks
Amsterdam, NetherlandsLeads the design, development, and optimization of large-scale backend systems for Delta Lake write-path operations. The role requires extensive distributed-systems experience and includes architectural ownership, production reliability work, cross-team technical leadership, and mentorship.