Research Engineer, Infrastructure, Inference

What You’ll Do

• Work alongside researchers and engineers to bring cutting-edge AI models into production.
• Collaborate with research teams to enable high-performance inference for novel architectures.
• Design and implement new techniques, tools, and architectures that improve performance, latency, throughput, and efficiency.
• Optimize our codebase and compute fleet (e.g., GPUs) to fully utilize hardware FLOPs, bandwidth, and memory.
• Extend orchestration frameworks (e.g., Kubernetes, Ray, SLURM) for distributed inference, evaluation, and large-batch serving.
• Establish standards for reliability, observability, and reproducibility across the inference stack.
• Publish and share learnings through internal documentation, open-source libraries, or technical reports that advance the field of scalable AI infrastructure.

Skills and Qualifications

Minimum qualifications:

• Bachelor’s degree or equivalent experience in computer science, engineering, or similar.
• Understanding of deep learning frameworks (e.g., PyTorch, JAX) and their underlying system architectures.
• Experience with inference serving systems optimized for throughput and latency (e.g., SGLang, vLLM).
• Thrive in a highly collaborative environment involving many, different cross-functional partners and subject matter experts.
• A bias for action with a mindset to take initiative to work across different stacks and different teams where you spot the opportunity to make sure something ships.
• Strong engineering skills, ability to contribute performant, maintainable code and debug in complex codebases.

Preferred qualifications:

• Experience training or supporting large-scale language models with hundreds of billions of parameters or more.
• Understanding of distributed compute systems, GPU parallelism, and hardware-aware optimizations.
• Contributions to open-source ML or systems infrastructure projects (e.g., SGLang, vLLM, PyTorch, Triton, DeepSpeed, XLA).
• Track record of improving research productivity through infrastructure design or process improvements.

Logistics

Compensation: Depending on background, skills and experience, the expected annual salary range for this position is $350,000 - $475,000 USD.

Benefits: Generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.