As a Software Engineer on the Inference Stack team, you will build the distributed runtime that powers large-scale LLM inference. This role involves working across the stack, from developer experience to low-level infrastructure, and owning systems in production.
Build and own high-performance inference runtime for Voice AI models including STT, TTS, and voice agents. Design real-time systems with low tail latency, collaborate cross-team, and optimize model serving for production workloads. Requires CS degree and real-time systems experience.
Build in-house tooling for post-training custom ML models using advanced techniques like RL and finetuning. Requires deep expertise in transformer training, PyTorch distributed systems, parallelism strategies, GPU performance optimization, and HPC platforms.
Builds and operates internal AI agents and LLM-powered workflows to boost engineering productivity across code writing, PR reviews, debugging, and documentation. Evaluates and deploys cutting-edge AI coding tools tailored to company needs.
Builds performance benchmarking, diagnostic, and optimization tools for LLM inference on GPU clusters. Early-career role requiring Python proficiency, systems curiosity, and interest in AI hardware—no prior experience needed.
Architects and leads development of scalable ML training infrastructure, including scheduling, storage, networking, and reinforcement learning systems. Requires proficiency in Go, Kubernetes expertise, distributed systems knowledge, and experience with cloud providers and ML workloads.
Software Engineer optimizes ML model inference performance using techniques like quantization and speculative decoding. Requires backend experience with PyTorch, TensorRT, CUDA, and deep GPU knowledge for LLMs.
As a Software Engineer on the Inference Stack team, you will build the distributed runtime that powers large-scale LLM inference. This role involves working across the stack, from developer experience to low-level infrastructure, and owning systems in production.
Build and own high-performance inference runtime for Voice AI models including STT, TTS, and voice agents. Design real-time systems with low tail latency, collaborate cross-team, and optimize model serving for production workloads. Requires CS degree and real-time systems experience.
Build in-house tooling for post-training custom ML models using advanced techniques like RL and finetuning. Requires deep expertise in transformer training, PyTorch distributed systems, parallelism strategies, GPU performance optimization, and HPC platforms.
Builds and operates internal AI agents and LLM-powered workflows to boost engineering productivity across code writing, PR reviews, debugging, and documentation. Evaluates and deploys cutting-edge AI coding tools tailored to company needs.
Builds performance benchmarking, diagnostic, and optimization tools for LLM inference on GPU clusters. Early-career role requiring Python proficiency, systems curiosity, and interest in AI hardware—no prior experience needed.
Architects and leads development of scalable ML training infrastructure, including scheduling, storage, networking, and reinforcement learning systems. Requires proficiency in Go, Kubernetes expertise, distributed systems knowledge, and experience with cloud providers and ML workloads.
Software Engineer optimizes ML model inference performance using techniques like quantization and speculative decoding. Requires backend experience with PyTorch, TensorRT, CUDA, and deep GPU knowledge for LLMs.