Skip to content
DatabricksDatabricksSan Francisco, CA

Senior Software Engineer, Foundation Model API

Build and shape the Foundation Model API serving layer for large-scale LLM inference at Databricks, bridging technical execution with product strategy for partner and self-hosted models. Requires 8+ years backend/infra engineering experience with distributed systems, real-time serving, and strong product ownership.

160k – 225k/yr
On-site8+ YOEML Engineering

About the role

Impact

  • Build LLM infrastructure powering large-scale inference workloads for customers through partner models (OpenAI, Anthropic, Gemini) and self-hosted models (Qwen, GPT-OSS, Llama)
  • Shape the direction of the FMAPI product — from roadmap to execution — by leveraging deep customer empathy and direct engagement with enterprise users and model providers
  • Improve reliability, latency, and efficiency of distributed AI workloads
  • Collaborate with platform, infra, and ML teams to deliver seamless end-to-end experiences
  • Shape how developers and data scientists build and interact with AI on Databricks

Requirements

  • 8+ years of experience in backend or infrastructure engineering
  • Experience with distributed systems, scalable APIs, or cloud-native infrastructure
  • Strong product and ownership mindset, with a focus on shipping user-facing value
  • Experience with real-time serving, ML infrastructure, or GPU orchestration
  • Familiarity with service-oriented architecture, deployment pipelines, and system observability
  • Strong programming skills in Scala, Go, or Python

Nice-to-Haves

  • Exposure to platforms like SageMaker, Vertex AI, or Azure ML
  • Built products that support AI workflows

Skills

Distributed SystemsScalable ApisCloud Native InfrastructureReal-Time ServingML InfrastructureGpu OrchestrationService-Oriented ArchitectureDeployment PipelinesSystem ObservabilityScalaGoPythonSageMakerVertex AiAzure Ml

Similar roles

ML Engineering jobs
Pearl Health

Senior Software Engineer

Pearl HealthSeattle, WA +2

Senior full-stack engineer building production Applied AI features (RAG pipelines, Agentic workflows, LLM integrations) for Pearl Health's value-based care platform. Requires 7+ years software engineering experience, hands-on LLM/RAG expertise, Python, cloud, and mentoring skills.

160k – 200k/yr
Remote7+ YOEML Engineering
SpotOn

Senior Software Engineer

SpotOnChicago, IL +3

Senior AI Engineer building agentic workflows and orchestration layers to automate manual business processes for operations, sales, and customer support teams. Requires 7+ years software engineering experience, production LLM/agent expertise, and strong Python/TypeScript skills.

160k – 190k/yr
Hybrid7+ YOEML Engineering
Honor

Senior Engineer, Applied AI

HonorUnited States

Senior Engineer building production-grade applied AI systems, LLM-powered tools, retrieval/memory patterns, and human-in-the-loop workflows to create organizational intelligence for home care operations. Requires Python experience, production system design skills, and cross-functional collaboration.

160k – 180k/yr
Remote5+ YOEML Engineering
Databricks

Senior Software Engineer, AI Runtime

DatabricksMountain View, CA +1

Senior Software Engineer building and scaling Databricks' managed GPU training platform (AI Runtime) for large-scale distributed AI model training. Requires 5+ years in distributed systems and hands-on experience with GPU training frameworks.

160k – 225k/yr
On-site5+ YOEML Engineering
Octus

Senior AI Engineer

OctusNew York, NY

Senior AI Engineer building and scaling production GenAI systems for CreditAI, including multi-agent workflows, RAG pipelines, LLM fine-tuning, and AWS deployments. Requires 5+ years of AI/ML experience and deep Python expertise.

160k – 170k/yr
On-site5+ YOEML Engineering