# Software Engineer - BIS

**Company:** [Baseten](https://hotfix.jobs/companies/baseten)
**Location:** San Francisco, CA
**Role:** ML Engineering
**Salary:** $180k – $360k/yr
**Skills:** Distributed Systems, Backend Infrastructure, Platform Engineering, Kubernetes, Dynamo, vLLM, Sglang, Tensorrt-Llm, Gpu Workloads, CI/CD
**Posted:** 2026-06-02

> As a Software Engineer on the Inference Stack team, you will build the distributed runtime that powers large-scale LLM inference. This role involves working across the stack, from developer experience to low-level infrastructure, and owning systems in production.

## Job Description

## RESPONSIBILITIES

* Develop infrastructure and orchestration systems for deploying and managing large-scale distributed LLM inference
* Work across the stack, from customer-facing features to low-level infrastructure components
* Build platform capabilities related to routing, autoscaling, scheduling, observability, and runtime management
* Improve the reliability, scalability, and usability of our inference stack
* Collaborate closely with Model Performance engineers to make new inference optimizations broadly available to customers and easy to configure
* Help define best practices around testing, release automation, benchmarking, and operational excellence
* Debug complex production systems spanning Kubernetes, distributed runtimes, networking, and GPU workloads
* Make thoughtful engineering tradeoffs balancing performance, reliability, operational simplicity, and developer experience
* Own projects end-to-end: from architecture and implementation through deployment, monitoring, and iteration based on customer feedback

## REQUIREMENTS

* Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or a related field
* Strong background in distributed systems, backend infrastructure, or platform engineering
* Experience building and operating production systems where reliability, latency, and scale are first-class concerns
* Strong sense of developer experience: you think about how systems are used, not just how they work
* Motivated and willing to learn new languages, frameworks, and systems as needed
* Ability to debug complex systems across multiple layers of the stack
* Genuine interest in inference engineering. You don’t need to have hands on experience but are willing to learn
* Excellent communication and collaboration skills

## BONUS

* Experience with Kubernetes, including concepts like operators and custom resources
* Prior work on Dynamo, vLLM, SGLang, TensorRT-LLM, or similar inference frameworks
* Experience with distributed scheduling, autoscaling, or service orchestration
* Experience operating GPU workloads in production
* Familiarity with observability tooling, CI/CD systems, or release automation
* Experience contributing to open-source infrastructure or ML systems

## BENEFITS

* Competitive compensation, including meaningful equity.
* 100% coverage of medical, dental, and vision insurance for employee and dependents
* Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)
* Paid parental leave
* Fertility and family-building stipend through Carrot
* Company-facilitated 401(k)
* Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.

## Similar roles

- [Software Engineer, AI Platform](https://hotfix.jobs/jobs/7ddad000-3e1c-47cb-a45b-31d8e72abc4f) - Notion - San Francisco, CA - $180k – $201k/yr
- [Research Engineer, Generalist](https://hotfix.jobs/jobs/1308107a-d911-4d6a-924d-e87d4654d356) - Exa - San Francisco, CA - $180k – $350k/yr
- [AI Engineer (Core)](https://hotfix.jobs/jobs/36b94901-fb9d-47d4-a46b-51ef8679ea67) - Build - San Francisco, CA - $180k – $250k/yr
- [Research Engineer](https://hotfix.jobs/jobs/54931695-a06a-4671-b30e-cacd5daf8c92) - Lightning AI - Remote - $180k – $250k/yr
- [Applied AI Engineer](https://hotfix.jobs/jobs/57f48e1d-2be5-4629-8d24-f5987223d169) - Rillet - San Francisco, CA - $180k – $240k/yr

**Apply:** https://hotfix.jobs/jobs/e9bc193b-be22-4ba7-8730-9d1ea41c31ef
**Canonical:** https://hotfix.jobs/jobs/e9bc193b-be22-4ba7-8730-9d1ea41c31ef