Skip to content
DatabricksDatabricksSan Francisco, CA

Staff Backend Software Engineer- (AI Platform)

Staff Backend Engineer building and scaling Databricks Foundation Model Serving APIs for frontier LLMs (Llama, Claude, GPT). Design high-throughput low-latency GPU inference systems, influence architecture, and mentor teams. 10+ years large-scale distributed systems experience required; no prior ML/AI needed.

192k – 260k/yr
On-site10+ YOEBackend Engineering

About the role

Impact

  • Design and implement core systems and APIs that power Databricks Foundation Model Serving, ensuring scalability, reliability, and operational excellence.
  • Partner with product and engineering leadership to define the technical roadmap and long-term architecture for serving workloads.
  • Drive architectural decisions and trade-offs to optimize performance, throughput, autoscaling, and operational efficiency for GPU serving workloads.
  • Contribute directly to key components across the serving infrastructure — from working in systems like vLLM and SGLang to creating token based rate limiters and optimizers — ensuring smooth and efficient operations at scale.
  • Collaborate cross-functionally with product, platform, and research teams to translate customer needs into reliable and performant systems.
  • Establish best practices for code quality, testing, and operational readiness, and mentor other engineers through design reviews and technical guidance.
  • Represent the team in cross-organizational technical discussions and influence Databricks’ broader AI platform strategy.

Requirements

  • 10+ years of experience building and operating large-scale distributed systems.
  • Experience leading high-scale operationally sensitive backend systems.
  • A track record of up-leveling teams engineering excellence.
  • Strong foundation in algorithms, data structures, and system design as applied to large-scale, low-latency serving systems.
  • Proven ability to deliver technically complex, high-impact initiatives that create measurable customer or business value.
  • Strong communication skills and ability to collaborate across teams in fast-moving environments.
  • Strategic and product-oriented mindset with the ability to align technical execution with long-term vision.
  • Passion for mentoring, growing engineers, and fostering technical excellence.

Nice-to-Haves

  • Interest in building LLM APIs and runtimes at scale.
  • Experience with systems like vLLM and SGLang.
  • Experience with GPU workloads, token based rate limiters, and optimizers.

Skills

Distributed SystemsSystem Designlow-latency servingvLLMsglanggpu workloadsAPIsAlgorithmsData Structuresautoscaling

Similar roles

Backend Engineering jobs
Hinge Health

Staff Software Engineer, Device Software

Hinge HealthSan Francisco, CA

Staff-level backend engineer leading device-adjacent software systems at Hinge Health. Owns Python/Next.js services, APIs, and integrations connecting devices, mobile/web clients, and backend infrastructure.

192k – 288k/yr
Hybrid8+ YOEBackend Engineering
1Password

Staff Developer, Identity

1PasswordUnited States

Lead architect and developer for 1Password's core identity platform, designing scalable authentication, authorization, and access management systems using Go. Requires 8+ years of experience, deep Go expertise, and 5+ years in identity and access management.

192k – 278k/yr
Remote8+ YOEBackend Engineering
Illumio

Staff Engineer - Backend Java

IllumioSunnyvale, CA

Designs and deploys scalable cloud architectures using Kubernetes and major cloud platforms (AWS/Azure/GCP). Collaborates cross-functionally to optimize for security, performance, and resilience in cybersecurity-focused backend systems. Requires strong Java/Python experience building distributed systems.

192k – 230k/yr
On-siteBackend Engineering
1Password

Staff Developer, Growth

1PasswordUnited States

Staff Developer builds self-serve growth experiences, experimentation platforms, and integrations for B2B products. Requires 8+ years SaaS experience, Go proficiency, and leading cross-team initiatives in frontend, backend, and data systems.

192k – 278k/yr
Remote8+ YOEBackend Engineering
Databricks

Staff Software Engineer - Database Engine Internals

DatabricksMountain View, CA +1

Designs and implements next-gen query engines and storage for Lakehouse architecture, focusing on query optimization, distributed execution, vectorized processing, and efficient storage. Requires 8+ years in database/distributed systems.

192k – 260k/yr
On-site8+ YOEBackend Engineering