# ML Model Serving Engineer

**Company:** [Sesame](https://hotfix.jobs/companies/sesame)
**Location:** San Francisco, CA, New York, NY, Bellevue, WA
**Role:** ML Engineering
**Salary:** $175k – $280k/yr
**Skills:** PyTorch, vLLM, Sglang, Kubernetes, Ray, GCP, AWS, Azure, Llm Serving, Performance Optimization
**Posted:** 2025-03-14

> Optimizes and extends ML model serving infrastructure for LLMs, speech, and vision models, focusing on high-throughput, low-latency inference using frameworks like VLLM and SGLang. Requires deep PyTorch expertise, systems programming, and performance engineering for reliable production deployment.

## Job Description

## Responsibilities
- Turbocharge our serving layer, consisting of a variety of LLM, speech, and vision models.
- Partner with ML infrastructure and training engineers to build a fast, cost-effective, accurate, and reliable serving layer to power a new consumer product category.
- Modify and extend LLM serving frameworks like VLLM and SGLang to take advantage of the latest techniques in high-performance model serving.
- Work with the training team to identify opportunities to produce faster models without sacrificing quality.
- Use techniques like in-flight batching, caching, and custom kernels to speed up inference.
- Find ways to reduce model initialization times without sacrificing quality.

## Required Qualifications
- Expert in some differentiable array computing framework, preferably **PyTorch**.
- Expert in optimizing machine learning models for serving reliably at high throughput, with low latency.
- Significant systems programming experience (e.g., working on high-performance server systems—comfortable with the internals of VLLM as with a complex PyTorch codebase).
- Significant performance engineering experience (e.g., bottleneck analysis in high-scale server systems or profiling low-level systems code).
- Always up to date on the latest techniques for model serving optimization.

## Preferred Qualifications
- Familiarity with high-performance LLM serving (e.g., experience with VLLM, SGlang deployment, and internals).
- Experience with a public cloud platform such as **GCP**, **AWS**, or **Azure**.
- Experience deploying and scaling inference workloads in the cloud using **Kubernetes**, **Ray**, etc.
- Track record of leading complex multi-month projects without assistance.

## Benefits
- 401(k) max employer match: 3.5% of compensation
- 100% employer-paid health, vision, and dental benefits for you and your dependents
- Unlimited PTO and sick time
- Flexible spending account with employer matching up to $1,650/year (medical FSA)
- Guardian Employee Assistance Program (EAP)
- Competitive stock options

## Similar roles

- [ML Engineer, Agentic Systems](https://hotfix.jobs/jobs/fb7b9e6d-de90-4df1-93b5-b35cfc715143) - Mirage - New York, NY - $175k – $275k/yr
- [Software Engineer, ML Products](https://hotfix.jobs/jobs/9fbada47-05ec-4c0c-84dc-31c53d8a0541) - Mirage - New York, NY - $175k – $275k/yr
- [Robotic Software Engineer, Perception](https://hotfix.jobs/jobs/9cb99e86-43d1-4920-8b0f-6352de40420b) - Applied Intuition - Sunnyvale, CA - $175k – $250k/yr
- [Research Engineer](https://hotfix.jobs/jobs/3ac0b434-bec7-4f49-93a3-787ad4af3951) - Hedra - San Francisco, CA - $175k – $275k/yr
- [Software Engineer, Applied AI](https://hotfix.jobs/jobs/df6cdae5-e0f7-41ee-a10f-11f4db42d654) - Auctor - New York, NY - $175k – $290k/yr

**Apply:** https://hotfix.jobs/jobs/5321aa17-8373-48e6-9901-d75115dc9049
**Canonical:** https://hotfix.jobs/jobs/5321aa17-8373-48e6-9901-d75115dc9049