# Senior Software Engineer, Model Serving

**Company:** [Databricks](https://hotfix.jobs/companies/databricks)
**Location:** San Francisco, CA
**Role:** ML Engineering
**Salary:** $166k – $225k/yr
**Experience:** 5+ years
**Skills:** Distributed Systems, Model Serving, Inference Systems, System Design, Kubernetes, Autoscaling, Observability, Routing, Caching, GPU
**Posted:** 2026-01-30

> Designs and builds scalable infrastructure for high-throughput, low-latency AI/ML model serving on CPU/GPU. Requires 5+ years in distributed systems, inference expertise, and strong system design skills.

## Job Description

## The impact you will have:
- Design and implement core systems and APIs that power Databricks Model Serving, ensuring scalability, reliability, and operational excellence.
- Drive architectural decisions and trade-offs to optimize performance, throughput, autoscaling, and operational efficiency for CPU and GPU serving workloads.
- Contribute directly to key components across the serving infrastructure — from model container builds and deployment workflows to runtime systems like routing, caching, observability, and intelligent autoscaling — ensuring smooth and efficient operations at scale.
- Collaborate cross-functionally with product, platform, and research teams to translate customer needs into reliable and performant systems.
- Lead technical initiatives that improve latency, availability, and cost-effectiveness across both customer-facing and foundational serving layers.
- Establish best practices for code quality, testing, and operational readiness, and mentor other engineers through design reviews and technical guidance.

## What we look for:
- 5+ years of experience building and operating large-scale distributed systems.
- Experience in model serving, inference systems, or related infrastructure (e.g., routing, scheduling, autoscaling, and observability).
- Strong foundation in algorithms, data structures, and system design as applied to large-scale, low-latency serving systems.
- Proven ability to deliver technically complex, high-impact initiatives that create measurable customer or business value.
- Experience building architecture for large-scale, performance-sensitive CPU/GPU inference systems.
- Strong communication skills and ability to collaborate across teams in fast-moving environments.
- Customer-focused mindset with the ability to align implementation details with product goals.
- Passion for mentoring, growing engineers, and fostering technical excellence.

## Similar roles

- [Senior Machine Learning Engineer - GenAI Platform](https://hotfix.jobs/jobs/56b930a9-1d7c-427a-a3df-a38a7a5021f6) - Databricks - San Francisco, CA - $166k – $225k/yr
- [Senior Machine Learning Operations Engineer](https://hotfix.jobs/jobs/d3b3a89a-90b9-4f8c-995d-168d4de2ff75) - Mercury - San Francisco, CA - $167k – $208k/yr
- [Senior Applied Research Engineer](https://hotfix.jobs/jobs/9273299c-3014-4320-98e1-ae4b9d661aad) - Drata - San Francisco, CA - $167k – $226k/yr
- [Senior AI Engineer, Agent Harness](https://hotfix.jobs/jobs/c5e7d0e4-19a7-4938-aa4c-9ef179acdd19) - Drata - San Francisco, CA - $167k – $226k/yr
- [Senior GenAI Software Engineer](https://hotfix.jobs/jobs/9ae4d14f-3a6a-4a56-95dd-90b1acfe9fdc) - Liftoff - Remote - $165k – $230k/yr

**Apply:** https://hotfix.jobs/jobs/b9f85503-574d-49d4-aa09-a2560df24787
**Canonical:** https://hotfix.jobs/jobs/b9f85503-574d-49d4-aa09-a2560df24787