Skip to content
DatabricksDatabricksSeattle, WA

Staff Software Engineer

Staff engineer driving technical vision and architecture for Spark Structured Streaming. Build core capabilities like advanced state management and operators; improve latency, throughput, and cost. Requires 8+ years in big-data, Spark, or database systems plus passion for distributed systems.

$182k – $247k/yr
On-site8+ YOEML Engineering

About the job

What you'll do

  • Set and drive the technical vision for Spark Structured Streaming across OSS and the Databricks Data Intelligence Platform
  • Design and build core engine capabilities — state management, new operators, and architectural improvements to latency and throughput
  • Raise the bar for engineering quality and operability, building software that is not just high quality but easy to run in production
  • Make company-wide impact by driving stream processing adoption across the Databricks product portfolio
  • Guide long-term architecture and technical-debt decisions, balancing them against the product roadmap
  • Mentor and technically lead engineers on the team, and partner with the Engineering Manager to attract and grow top-tier talent

What we look for

  • BS (or higher) in Computer Science or a related technical field, or equivalent practical experience
  • 8+ years building related systems — big-data ecosystems, Apache Spark, or database internals
  • A passion for database systems, storage systems, distributed systems, language design, or performance optimization
  • Comfortable working toward a multi-year vision with incremental deliverables
  • A track record of delivering features while maintaining a high bar for operational excellence and engineering quality
  • Comfortable working cross-functionally with product management and directly with customers, with the ability to deeply understand the product and customer personas

Skills

SparkDistributed SystemsDatabase InternalsState ManagementBig DataPerformance OptimizationLanguage DesignStorage Systems

Similar jobs

ML Engineering jobs
Nuro

Senior/Staff Engineer, Machine Learning - Online Mapping

NuroMountain View, CA

Develops state-of-the-art online mapping models and algorithms for autonomous vehicles using real sensor data. Requires expertise in ML, robotics, computer vision, and deep learning frameworks like PyTorch.

$184k – $334k/yrOn-siteML Engineering
OnePay

Staff Applied Scientist, Personalization

OnePayUnited States

Design and productionize ML, deep learning, and LLM models for personalization, recommendations, and search systems. Requires 7+ years building production ML/AI with business impact and strong cross-functional collaboration skills.

$180k – $250k/yrRemote7+ YOEML Engineering
Black Forest Labs

Member of Technical Staff - Model Serving / API Backend Engineer

Black Forest LabsSan Francisco, CA

Build and optimize high-performance ML inference services and APIs that turn frontier research models (FLUX, Stable Diffusion) into production systems serving millions of requests. Requires experience scaling ML serving infrastructure, GPU optimization, and production backend systems.

$180k – $300k/yrHybrid5+ YOEML Engineering
xAI

Member of Technical Staff

xAIPalo Alto, CA

Build and optimize the RL training framework and infrastructure for large-scale workloads at SpaceXAI, from ablations to production runs. Requires experience with distributed systems and proficiency in Python, JAX, Rust, or C++.

$180k – $440k/yrOn-site5+ YOEML Engineering
LlamaIndex

Member of Technical Staff, Applied Research

LlamaIndexSan Francisco, CA

Build and productionize vision-language models for document understanding at LlamaIndex. Focus on training, fine-tuning, synthetic data, benchmarking, and turning research prototypes into accurate, low-latency production systems for real-world PDFs, tables, and enterprise docs. Requires 3+ years ML engineering/applied research experience with strong PyTorch skills.

$180k – $250k/yrHybrid3+ YOEML Engineering