Software Engineer, AI Gateway
Develop and enhance the AI Gateway platform, building unified APIs for AI models with rate limiting, failovers, and integrations. Requires 5+ years experience in JavaScript/TypeScript, backend, and distributed systems.
About the job
What You Will Do
- Contribute to the design, implementation, and maintenance of the AI Gateway platform, emphasizing features like unified API endpoints, rate limit management, and intelligent failover mechanisms to boost uptime and reliability.
- Write clean, efficient, and well-documented code, conducting thorough testing to ensure low-latency responses and stability for high-volume AI inference requests.
- Collaborate with cross-functional teams, including product managers, AI researchers, and infrastructure engineers, to integrate new AI providers and models while addressing scalability and performance challenges.
- Engage with the open-source community, contribute to AI SDK and related projects, and align with Vercel's ethos of fostering developer tools.
- Gather user feedback and analytics to drive innovations in AI Gateway, such as enhanced billing unification, provider-agnostic authentication, and analytics for usage insights.
About You
- You have at least 5+ years of relevant experience
- Strong proficiency in JavaScript/TypeScript and experience with backend development, APIs, and cloud infrastructure
- Experience with AI/ML integrations, API gateways, distributed systems, or handling high-throughput services (e.g., rate limiting, caching, failovers)
- Experience contributing to or participating in open source projects
- Excellent communication skills and the ability to work effectively in a collaborative team environment
Benefits
- Competitive compensation package, including equity.
- Inclusive Healthcare Package.
- Learn and Grow - we provide mentorship and send you to events that help you build your network and skills.
- Flexible Time Off.
- We will provide you the gear you need to do your role, and a WFH budget for you to outfit your space as needed.
San Francisco, CA base pay range: $196,000-$294,000. Actual salary will be based on job-related skills, experience, and location.
Skills
JavaScript, TypeScript, Ai/Ml Integrations, Api Gateways, Distributed Systems, Rate Limiting, Caching, Failovers, OpenAI, Anthropic
Similar jobs
ML Engineering jobs
Nuro
Mountain View, CABuild trustworthy infrastructure for production LLM agents, closed-loop evaluation, and autonomous research workflows. The role requires strong Python and distributed-systems experience, hands-on LLM post-training and inference knowledge, and experience operating agent systems at scale.
ClickUp
United States
Build and operate large-scale ranking and retrieval systems that power search relevance, including hybrid lexical/vector search, embeddings, query understanding, and permission-aware retrieval. Requires a bachelor's degree and 5+ years of ML engineering experience in ranking or information retrieval.
Atomicmachines
Emeryville, CA
Build and operate production ML infrastructure spanning training, deployment, serving, monitoring, data pipelines, and feedback-driven retraining. The role requires strong MLOps and DevOps experience, Python and SQL proficiency, and ownership of reliable cloud-based systems.
Tessera Labs
San Jose, CABuild and scale post-training, reinforcement-learning, evaluation, and inference systems for long-horizon agents operating over complex enterprise software. The role requires strong Python and PyTorch or JAX skills, distributed GPU experience, empirical rigor, and the ability to take research results into production.
Tessera Labs
San Jose, CABuild and operate production AI agents that transform enterprise processes, data, and code. The role focuses on tool layers, retrieval, context management, evaluations, monitoring, auditability, and guardrails, requiring strong Python and TypeScript plus experience with production LLM systems and traditional machine learning.