# Staff Software Engineer, Model Serving

**Company:** [Databricks](https://hotfix.jobs/companies/databricks)
**Location:** San Francisco, CA
**Role:** ML Engineering
**Salary:** $192k – $260k/yr
**Experience:** 10+ years
**Skills:** Model Serving, Inference Systems, Distributed Systems, Kubernetes, Autoscaling, Routing, Caching, Observability, GPU, System Design
**Posted:** 2026-01-30

> Designs and builds scalable, low-latency model serving infrastructure for AI/ML models across CPU/GPU workloads. Requires 10+ years in large-scale distributed systems and deep expertise in inference systems, architecture, and cross-team collaboration.

## Job Description

## The impact you will have:

- Design and implement core systems and APIs that power Databricks Model Serving, ensuring scalability, reliability, and operational excellence.
- Partner with product and engineering leadership to define the technical roadmap and long-term architecture for serving workloads.
- Drive architectural decisions and trade-offs to optimize performance, throughput, autoscaling, and operational efficiency for CPU and GPU serving workloads.
- Contribute directly to key components across the serving infrastructure — from model container builds and deployment workflows to runtime systems like routing, caching, observability, and intelligent autoscaling — ensuring smooth and efficient operations at scale.
- Collaborate cross-functionally with product, platform, and research teams to translate customer needs into reliable and performant systems.
- Lead technical initiatives that improve latency, availability, and cost-effectiveness across both customer-facing and foundational serving layers.
- Establish best practices for code quality, testing, and operational readiness, and mentor other engineers through design reviews and technical guidance.
- Represent the team in cross-organizational technical discussions and influence Databricks’ broader AI platform strategy.

## What we look for:

- **10+ years** of experience building and operating large-scale distributed systems.
- Deep expertise in model serving, inference systems, and related infrastructure (e.g., routing, scheduling, autoscaling, and observability).
- Strong foundation in algorithms, data structures, and system design as applied to large-scale, low-latency serving systems.
- Proven ability to deliver technically complex, high-impact initiatives that create measurable customer or business value.
- Experience leading architecture for large-scale, performance-sensitive CPU/GPU inference systems.
- Strong communication skills and ability to collaborate across teams in fast-moving environments.
- Strategic and product-oriented mindset with the ability to align technical execution with long-term vision.
- Passion for mentoring, growing engineers, and fostering technical excellence.

## Similar roles

- [Staff Software Engineer](https://hotfix.jobs/jobs/8ed0c488-5b98-4e7e-a7c1-058995ebc01f) - Databricks - New York, NY - $192k – $260k/yr
- [Staff Software Engineer, Foundational Model Serving](https://hotfix.jobs/jobs/83a6b20d-e859-4c06-8017-fde1640d595b) - Databricks - San Francisco, CA - $192k – $260k/yr
- [Staff Software Engineer - GenAI Performance and Kernel](https://hotfix.jobs/jobs/7d20b86b-17a6-4dbe-8bc0-89703822b0fc) - Databricks - San Francisco, CA - $191k – $233k/yr
- [Staff Software Engineer - GenAI inference](https://hotfix.jobs/jobs/c24ce5c3-c403-4ff3-90c0-c110271f0098) - Databricks - San Francisco, CA - $191k – $233k/yr
- [Senior/Staff Machine Learning Engineer, Sensor Simulation](https://hotfix.jobs/jobs/67882c92-6808-4924-b827-81f676c9980f) - Nuro - Mountain View, CA - $194k – $291k/yr

**Apply:** https://hotfix.jobs/jobs/d1f89e48-0d11-4e45-90a5-27d23a40d3d8
**Canonical:** https://hotfix.jobs/jobs/d1f89e48-0d11-4e45-90a5-27d23a40d3d8