# Senior Staff Software Engineer - Lakeflow Pipelines Datasets

**Company:** [Databricks](https://hotfix.jobs/companies/databricks)
**Location:** Mountain View, CA, San Francisco, CA
**Role:** Backend Engineering
**Salary:** $229k – $314k/yr
**Experience:** 10+ years
**Skills:** Distributed Systems, Streaming Systems, Database Internals, Storage Systems, Performance Engineering, Declarative Dataflow Processing, Resource Management, Stream Processing, Autonomic Computing, Spark
**Posted:** 2026-04-02

> Leads technical architecture and engineering for Lakeflow Pipelines Datasets platform, owning long-term direction for declarative pipelines, materialized views, and data infrastructure at exabyte scale. Requires 10+ years in large-scale distributed systems with deep expertise in databases, storage, or streaming.

## Job Description

## Responsibilities

- Architect and deliver a highly scalable, fault-tolerant platform for declarative pipelines and materialized views serving thousands of production customers.
- Turn ambiguous, large-scale technical challenges into clear execution plans that ship incrementally and hold up at exabyte scale.
- Bridge the gap between research and production by bringing ideas from the academic literature or internal R&D into systems that solve real customer problems.
- Lead deep systems work: performance diagnosis on large production clusters, low-level debugging, and optimization that moves the needle on cost and latency.
- Shape product direction alongside engineering and product leadership, owning technical strategy from design through delivery.
- Drive complex, multi-team technical initiatives that require sustained coordination and clear technical judgment.

## Requirements

- 10+ years of experience building, operating, and evolving large-scale distributed systems in production.
- Deep expertise in one or more of: database internals, storage systems, distributed computing, streaming systems, language/API design, or performance engineering.
- Track record of executing against a multi-year technical vision through well-sequenced incremental milestones.
- Strong algorithmic foundations with the practical judgment to know when they matter and when simpler solutions win.
- A bias toward customer impact over technical novelty for its own sake.
- Experience building alignment across teams and driving initiatives from conception through customer adoption.
- BS, MS, or PhD in Computer Science or a related field (or equivalent depth of experience).

## Similar roles

- [Sr. Staff Software Engineer - Observability, Insights & Governance](https://hotfix.jobs/jobs/f42d228d-863f-4006-a53c-0f3638d291b3) - Databricks - Mountain View, CA - $229k – $314k/yr
- [Sr. Staff Software Engineer - Managed Tables](https://hotfix.jobs/jobs/843cc956-e82b-4932-be11-08366aa0c5b7) - Databricks - Mountain View, CA - $229k – $314k/yr
- [Staff Software Engineer](https://hotfix.jobs/jobs/36503595-3254-45d1-9ad8-0c2da0246a85) - Carta - San Francisco, CA - $228k – $300k/yr
- [Staff Software Engineer](https://hotfix.jobs/jobs/a77b152b-1449-47f3-ae51-4bdc7baa3fcf) - Tines - Remote - $230k – $250k/yr
- [Software Architect](https://hotfix.jobs/jobs/8b901b0f-55af-4428-9aad-768257c998bd) - Machinify - Remote - $230k – $280k/yr

**Apply:** https://hotfix.jobs/jobs/e5435946-9d52-495e-8472-51fdd49df589
**Canonical:** https://hotfix.jobs/jobs/e5435946-9d52-495e-8472-51fdd49df589