Skip to content
DatabricksDatabricksSan Francisco, CA

Staff Software Engineer, Foundation Model API

Build and shape the Foundation Model API serving layer for large-scale LLM inference (partner and self-hosted models) at Databricks. Requires 8+ years backend/infra engineering experience with distributed systems, ML infrastructure, and a strong product ownership mindset.

190k – 265k/yr
On-site8+ YOEML Engineering

About the role

Impact

  • Build LLM infrastructure powering large-scale inference workloads for customers through partner models (OpenAI, Anthropic, Gemini) and self-hosted models (Qwen, GPT-OSS, Llama)
  • Shape the direction of the FMAPI product — from roadmap to execution — by leveraging deep customer empathy and direct engagement with enterprise users and model providers
  • Improve reliability, latency, and efficiency of distributed AI workloads
  • Collaborate with platform, infra, and ML teams to deliver seamless end-to-end experiences
  • Shape how developers and data scientists build and interact with AI on Databricks

Requirements

  • 8+ years of experience in backend or infrastructure engineering
  • Experience with distributed systems, scalable APIs, or cloud-native infrastructure
  • Strong product and ownership mindset, with a focus on shipping user-facing value
  • Experience with real-time serving, ML infrastructure, or GPU orchestration
  • Familiarity with service-oriented architecture, deployment pipelines, and system observability
  • Strong programming skills in Scala, Go, or Python

Nice-to-Haves

  • Exposure to platforms like SageMaker, Vertex AI, or Azure ML
  • Built products that support AI workflows

Skills

Distributed Systemsscalable apiscloud native infrastructurereal-time servingML Infrastructuregpu orchestrationservice-oriented architecturedeployment pipelinessystem observabilityScalaGoPythonSageMakervertex aiazure ml

Similar roles

ML Engineering jobs
Databricks

Staff Software Engineer - AI Research Infrastructure

DatabricksNew York, NY

Founding member of a new team building foundational evaluation infrastructure and flywheels for Databricks' AI/Genie Agents. Design scalable tooling for benchmarking, regression detection, and quality measurement that drives continuous agent improvement across research, training, and production.

190k – 270k/yr
On-site6+ YOEML Engineering
Databricks

Staff Software Engineer, AI Runtime

DatabricksMountain View, CA +1

Staff Software Engineer building and scaling Databricks' managed large-scale GPU training platform (AIR). Focus on distributed training performance, scheduling, fault tolerance, and developer experience for thousands of accelerators.

190k – 265k/yr
On-site10+ YOEML Engineering
Databricks

Staff Machine Learning Engineer

DatabricksSan Francisco, CA +1

Develops and deploys state-of-the-art GenAI models and systems for Databricks products like Assistant and Genie. Requires 2-8 years ML engineering experience, proficiency in Python/PyTorch/TensorFlow, and expertise in LLMs.

190k – 285k/yr
HybridML Engineering
Pinterest

Staff Machine Learning Engineer, Computer Vision

PinterestSan Francisco, CA

Staff Machine Learning Engineer developing state-of-the-art visual encoders and multimodal models at Pinterest Labs. Prototype visual reasoning tools, train billion-scale models on rich visual-text data, ship to production for recommender systems and VLMs, publish research, and mentor juniors. Requires strong CV/ML background, publications, and PhD or equivalent.

189k – 390k/yr
Hybrid7+ YOEML Engineering
Pinterest

Staff Machine Learning Engineer

PinterestSan Francisco, CA +1

As a Staff Machine Learning Engineer, you will lead the ML strategy and execution for the Advertiser and Seller Experience team, building recommendation systems and context foundations. This role requires deep expertise in recommendation systems and modern agentic AI to shape advertiser and seller workflows.

189k – 390k/yr
Hybrid7+ YOEML Engineering