AI Systems Research and Development Engineer – LLM Inference Systems & Optimization

Responsibilities

• Design and develop high-performance LLM inference systems across distributed serving, runtime systems, GPU execution, and performance-critical kernels.
• Improve inference latency, generation speed, throughput, memory efficiency, scalability, and cost.
• Develop techniques including speculative and parallel decoding, prefill/decode disaggregation, adaptive parallelism, continuous batching, scheduling, KV-cache management, quantization, and communication optimization.
• Build adaptive inference systems that optimize execution for new model architectures, hardware, workloads, and deployment environments.
• Apply AI-native approaches to profiling, bottleneck identification, configuration search, code generation, experimentation, debugging, and performance tuning.
• Identify high-impact systems problems, prototype solutions, and drive successful ideas from research through production.
• Design distributed inference strategies across GPUs and nodes, including tensor, sequence, pipeline, data, and expert parallelism.
• Develop multi-model serving, dynamic resource management, model loading and swapping, and workload-aware scheduling.
• Analyze and optimize GPU kernels and operators for attention, mixture-of-experts, communication, and other performance-critical components.
• Explore model-system co-design and post-training techniques for efficient inference.
• Profile and benchmark end-to-end workloads across compute, memory, communication, networking, scheduling, and model execution.
• Collaborate with research, infrastructure, and product teams to deploy innovations in production.
• Open-source and publish innovations through technical blogs and leading systems and machine learning conferences.

Requirements

• Bachelor’s degree in Computer Science, Electrical Engineering, or a related field.
• 5+ years of experience in LLM inference systems, distributed AI systems, GPU systems, or high-performance computing.
• Strong understanding of modern LLM inference architectures and large-scale model-serving tradeoffs.
• Hands-on experience with LLM inference and serving frameworks such as vLLM, SGLang, TensorRT-LLM, or similar systems.
• Experience designing, extending, or optimizing inference runtimes, including scheduling, batching, KV-cache management, distributed execution, parallelism, speculative decoding, or disaggregated serving.
• Strong understanding of GPU architectures and experience with CUDA, Triton, or similar GPU programming environments.
• Experience with performance-oriented libraries and frameworks such as CUTLASS, cuBLAS, cuDNN, or related technologies.
• Experience profiling and diagnosing end-to-end system performance using Nsight Systems, Nsight Compute, or equivalent tools.
• Ability to independently identify important problems, define technical questions, and drive solutions through ambiguity.
• Ability to work across model, runtime, distributed-system, and hardware layers and reason about end-to-end performance tradeoffs.
• Excellent communication and cross-functional collaboration skills.

Nice-to-haves

• Master’s degree or PhD.
• Experience using AI-native engineering approaches to accelerate software development, experimentation, debugging, optimization, or system adaptation.

Compensation

• Annual salary range: $236,000–$330,000.