Skip to content
DatabricksDatabricksBellevue, WA

Senior Software Engineer - Distributed Data Systems

Build distributed data storage and processing systems for big data workloads including Apache Spark, Delta Lake, and performance optimization. Requires 5+ years in Java/Scala/C++, strong algorithms knowledge, and distributed systems experience.

158k – 214k
On-site5+ YOEData Engineering

About the role

Responsibilities

  • Build next generation distributed data storage and processing systems that outperform SQL query engines while supporting ETL, data science, and other workloads.
  • Example projects:
    • Apache Spark™: Develop the open source standard framework for big data.
    • Data Plane Storage: Build reliable, high-performance services and client libraries for cloud storage (AWS S3, Azure Blob Store).
    • Delta Lake: Create storage management system with ACID transactions, time travel, data lake scale, and warehouse performance.
    • Delta Pipelines: Orchestrate and operate tens of thousands of data pipelines with higher-level abstractions.
    • Performance Engineering: Develop fast, scalable query optimizer and execution engine.

Requirements

  • BS (or higher) in Computer Science or related field, or equivalent experience.
  • 5+ years production experience in Java, Scala, or C++.
  • Strong foundation in algorithms, data structures, and real-world applications.
  • Experience with distributed systems, databases, and big data systems (Apache Spark, Hadoop).
  • Comfortable working toward multi-year vision with incremental deliverables.
  • Motivated by customer value and impact.

Skills

JavaScalaC++SparkDelta LakeAws S3Azure Blob StorageAlgorithmsData StructuresDistributed Systems

Similar roles

Data Engineering jobs
Launchdarkly

Senior Backend Engineer, Data Platform

LaunchdarklyAustin, TX +8

Senior Backend Engineer building and operating distributed data infrastructure for streaming, batch processing, analytical stores, and warehouse exports at scale. Requires 6+ years backend experience with data pipelines, distributed systems, and production ownership (on-call).

158k – 257k
Remote6+ YOEData Engineering
PointClickCare

Senior Software Engineer- Big Data & Java

PointClickCareUnited States

Develops and maintains big data applications using Java and distributed systems like Apache Hudi, Spark, and Trino. Collaborates on requirements, ensures scalable code with TDD, and works with storage systems like S3 and ADLS.

158k – 176k
RemoteData Engineering
Checkr

Senior Data Engineer

CheckrDenver, CO +1

Senior Data Engineer building and scaling Checkr's centralized People Data platform and large-scale batch/streaming pipelines that power identity and people records for all products. Requires 5+ years experience with Python, PySpark, SQL, Spark, Kafka and AWS data services; leads execution, mentors, and partners with cross-functional teams.

157k – 218k
Hybrid5+ YOEData Engineering
Metropolis

Senior Data Engineer

MetropolisNew York, NY +4

Designs and owns data architecture, builds/optimizes pipelines and models using Python, SQL, Snowflake, dbt, and AWS. Manages SLAs, data governance, and mentors team; requires 5+ years experience and bachelor's in CS or related.

160k – 220k
On-site5+ YOEData Engineering
Lithic

Senior Software Engineer, Data Platform

LithicUnited States

Builds backend services and REST APIs serving data from Snowflake and other sources to internal teams, while supporting data pipelines with Airflow, Airbyte, and dbt. Requires strong Python, SQL, API design experience; preferred fintech background.

160k – 200k
RemoteData Engineering