# Staff Backend Software Engineer- (AI Platform)

**Company:** [Databricks](https://hotfix.jobs/companies/databricks)
**Location:** San Francisco, CA
**Role:** Backend Engineering
**Salary:** $192k – $260k/yr
**Experience:** 10+ years
**Skills:** Distributed Systems, System Design, low-latency serving, vLLM, sglang, gpu workloads, APIs, Algorithms, Data Structures, autoscaling
**Posted:** 2026-07-17

> Staff Backend Engineer building and scaling Databricks Foundation Model Serving APIs for frontier LLMs (Llama, Claude, GPT). Design high-throughput low-latency GPU inference systems, influence architecture, and mentor teams. 10+ years large-scale distributed systems experience required; no prior ML/AI needed.

## Job Description

## Impact
- Design and implement core systems and APIs that power Databricks Foundation Model Serving, ensuring scalability, reliability, and operational excellence.
- Partner with product and engineering leadership to define the technical roadmap and long-term architecture for serving workloads.
- Drive architectural decisions and trade-offs to optimize performance, throughput, autoscaling, and operational efficiency for GPU serving workloads.
- Contribute directly to key components across the serving infrastructure — from working in systems like vLLM and SGLang to creating token based rate limiters and optimizers — ensuring smooth and efficient operations at scale.
- Collaborate cross-functionally with product, platform, and research teams to translate customer needs into reliable and performant systems.
- Establish best practices for code quality, testing, and operational readiness, and mentor other engineers through design reviews and technical guidance.
- Represent the team in cross-organizational technical discussions and influence Databricks’ broader AI platform strategy.

## Requirements
- 10+ years of experience building and operating large-scale distributed systems.
- Experience leading high-scale operationally sensitive backend systems.
- A track record of up-leveling teams engineering excellence.
- Strong foundation in algorithms, data structures, and system design as applied to large-scale, low-latency serving systems.
- Proven ability to deliver technically complex, high-impact initiatives that create measurable customer or business value.
- Strong communication skills and ability to collaborate across teams in fast-moving environments.
- Strategic and product-oriented mindset with the ability to align technical execution with long-term vision.
- Passion for mentoring, growing engineers, and fostering technical excellence.

## Nice-to-Haves
- Interest in building LLM APIs and runtimes at scale.
- Experience with systems like vLLM and SGLang.
- Experience with GPU workloads, token based rate limiters, and optimizers.

## Similar roles

- [Staff Software Engineer, Device Software](https://hotfix.jobs/jobs/0a327d76-0bd9-4eec-92c4-5524d0010a55) - Hinge Health - San Francisco, CA - $192k – $288k/yr
- [Staff Developer, Identity](https://hotfix.jobs/jobs/56e246e2-b80e-482e-b0fe-8241e8b7e006) - 1Password - Remote - $192k – $278k/yr
- [Staff Engineer - Backend Java](https://hotfix.jobs/jobs/2077c009-80c2-44b6-b481-4ad07ba9ec81) - Illumio - Sunnyvale, CA - $192k – $230k/yr
- [Staff Developer, Growth](https://hotfix.jobs/jobs/9015df0f-1d12-4d89-a869-27143205bb9a) - 1Password - Remote - $192k – $278k/yr
- [Staff Software Engineer - Database Engine Internals](https://hotfix.jobs/jobs/5cf6daf3-dd3a-4ee8-980e-167089279cb9) - Databricks - Mountain View, CA - $192k – $260k/yr

**Apply:** https://hotfix.jobs/jobs/37655e0d-9bd8-4c24-a721-7df6fcbe8b86
**Canonical:** https://hotfix.jobs/jobs/37655e0d-9bd8-4c24-a721-7df6fcbe8b86