Staff Backend Software Engineer
190k – 261kNew York, NYOnsite8+ YOE
Summary
Builds LLM infrastructure for AI workloads, improves reliability and efficiency of distributed systems, and collaborates with ML teams. Requires 8+ years backend experience with Scala, Go, or Python, and expertise in distributed systems.
About the role
The impact you will have:
- Build LLM infrastructure powering trillions of tokens per week for customers through partner models (OpenAI, Anthropic, Gemini) and self hosted models (Qwen, GPT-OSS, Llama)
- Improve reliability, latency, and efficiency of distributed AI workloads
- Collaborate with platform, infra, and ML teams to deliver seamless end-to-end experiences
- Shape how developers and data scientists build and interact with AI on Databricks
What we look for:
- 8+ years of experience in backend or infrastructure engineering
- Strong programming skills in Scala, Go, or Python
- Experience with distributed systems, scalable APIs, or cloud-native infrastructure
- Familiarity with service-oriented architecture, deployment pipelines, and system observability
- Strong product and ownership mindset — you care about building the right solution, not just any solution
Bonus points for:
- Experience with real-time serving, ML infrastructure, or GPU orchestration
- Exposure to platforms like SageMaker, Vertex AI, or Azure ML
- Contributions to OSS projects like MLflow, PyTorch, Ray, vLLM, SGLang
- Built developer platforms or internal tools supporting AI workflows
Skills
ScalaGoPythonDistributed SystemsScalable APIsCloud-Native InfrastructureService-Oriented ArchitectureMLflowPyTorchRay
Similar roles at this salary range
All Backend Engineering jobs →Staff Software Engineer, Growth AI
Staff Software Engineer anchoring AI-powered growth products across SEO and exploratory teams. Architect production ML systems, partner with ML orgs, and set technical direction as a senior IC.
208k – 365kSan Francisco, CA +3Backend EngineeringHybridJavaLLMs
Staff Backend Engineer
Staff Backend Engineer to architect and scale voice AI systems for 911 centers, optimizing realtime pipelines and integrating with customer systems. Requires 6+ years of experience and strong engineering fundamentals.
180k – 240kSeattle, WABackend EngineeringOn-siteGoSQL