Staff Software Engineer - Data Platform
Leads the design, operation, and evolution of Databricks’ cross-company Data Intelligence Platform, including large-scale data systems, pipelines, governance, and infrastructure. Requires 10+ years of distributed-systems experience and substantial technical leadership on production data platforms.
About the job
Responsibilities
- Design and run the cross-company Data Intelligence Platform containing business and product metrics used to operate Databricks.
- Develop tooling and infrastructure to manage and run Databricks on Databricks at scale across multiple clouds, geographies, and deployment types.
- Build CI/CD processes, pipeline test frameworks, data-quality tooling, and infrastructure-as-code systems.
- Partner with engineering teams to develop the long-term vision and requirements for the Databricks product.
- Build reliable data pipelines and solve data problems using Databricks, partner products, and open-source tools.
- Provide early feedback on the design and operation of data products.
- Represent Databricks at academic and industry conferences and events.
- Provide technical leadership, mentor engineers, and align engineering execution with long-term product and company goals.
Requirements
- BS, MS, or PhD in Computer Science or a related major, or equivalent experience.
- 10+ years of industry experience building and operating large-scale distributed systems.
- 4+ years providing technical leadership on planet-scale production data systems.
- Experience leading architecture efforts for performance-sensitive systems, such as latency-critical services, multi-tenant platforms, or large-scale indexing pipelines.
- Strong communication skills and the ability to work effectively in fast-paced, cross-functional environments.
- Strategic mindset with the ability to connect engineering execution to longer-term product and company goals.
- Passion for mentoring and developing engineers.
Benefits
- Comprehensive benefits and perks designed to meet employees' needs, with region-specific details available from Databricks.
Skills
Databricks, Data Pipelines, Distributed Systems, CI/CD, Data Quality, Infrastructure As Code, Multi-Cloud Systems, Data Governance, Metric Stores, Data Transformation, Orchestration, Large-Scale Indexing Pipelines
Similar jobs
Data Engineering jobs
Celonis
Bengaluru, India
Leads the design and development of scalable analytic data infrastructure, including the foundation for Celonis’s Digital Twin. Requires 10+ years of analytics or data engineering experience, enterprise Databricks expertise, and strong stakeholder communication.
Vanta
Remote
Leads the re-platforming of Vanta’s compliance data layer from MongoDB to schema-aware PostgreSQL across high-throughput Kafka and S3 pipelines. The role requires staff-level distributed systems expertise, migration leadership, and strong experience with relational and document data modeling.
Databricks
Bengaluru, India
Leads the design and operation of Databricks’ large-scale Data Intelligence Platform, including metrics stores, ETL frameworks, multi-cloud pipelines, governance, and infrastructure tooling. Requires extensive industry experience, distributed-systems expertise, and technical leadership across complex data infrastructure initiatives.
Alpaca
Remote
Build and operate scalable lakehouse infrastructure, streaming and CDC pipelines, query systems, and self-serve BI capabilities. Requires 5+ years of data engineering experience, strong Kubernetes and infrastructure-as-code expertise, and hands-on experience with distributed data platforms.
StarTree
India
Build and operate distributed systems powering Apache Pinot’s real-time analytics platform at massive scale. The role requires strong distributed-systems expertise, end-to-end delivery ownership, and a focus on reliability, observability, and performance.