# Software Engineer, Model Routing & Inference

**Company:** [Cursor](https://hotfix.jobs/companies/cursor)
**Location:** New York, NY, San Francisco, CA
**Role:** Backend Engineering
**Skills:** Distributed Systems, Inference Serving, Traffic Routing, Real-Time Data Pipelines, Gpu Utilization, Api Gateways, Failover Systems, Backpressure, Admission Control, Capacity Planning
**Posted:** 2026-04-07

> Build the inference platform powering all AI interactions, focusing on high-throughput, low-latency distributed systems for model routing, failover, and cost/performance optimization at massive scale. Requires strong engineering fundamentals for production systems handling millions of requests.

## Job Description

## Responsibilities
- Build and evolve the inference gateway, a single abstraction over every provider's API semantics, so model onboarding becomes a config change.
- Design intelligent cross-provider failover so no single provider outage causes user-visible degradation.
- Design routing backpressure and admission control so traffic spikes don't cascade into providers.

## Requirements
- Deep experience building high-throughput, low-latency distributed systems, especially in inference serving, traffic routing, or real-time data pipelines.
- Comfortable reasoning about cost/performance tradeoffs at scale (GPU utilization, provider economics, capacity planning).
- Strong software engineering fundamentals and enjoy shipping production systems that handle millions of requests.
- Make good calls in the gray area: weighing reliability, cost, latency, and user experience when there isn't a single \"right\" answer.

## Similar roles

- [Software Engineer (Backend), Deliverability](https://hotfix.jobs/jobs/2784834d-42ef-4b0d-bb3c-b825cbae7c53) - Front - San Francisco, CA - $178k – $204k/yr
- [Backend Engineer](https://hotfix.jobs/jobs/b1af6614-7e82-451b-b33a-6e7723bb589c) - Scribe - San Francisco, CA - $140k – $185k/yr
- [Software Engineer, Codex -Enterprise Controls](https://hotfix.jobs/jobs/9fc148ac-afcb-4558-a41e-a772e8869a71) - OpenAI - San Francisco, CA - $230k – $385k/yr
- [Software Engineer III](https://hotfix.jobs/jobs/a91a8070-ade4-411a-a538-be8c46e121ab) - ZoomInfo - Bethesda, MD - $112k – $176k/yr
- [Software Engineer, Upstart Bank](https://hotfix.jobs/jobs/5dbad557-7d4b-47a1-a575-68c16e632c5f) - Upstart - Remote - $14k – $187k/yr

**Apply:** https://hotfix.jobs/jobs/71758868-f58e-4613-8baf-3f61d4fd41bd
**Canonical:** https://hotfix.jobs/jobs/71758868-f58e-4613-8baf-3f61d4fd41bd