AI Engineer - Model Performance

Responsibilities

• Own inference performance: implement speculative decoding, quantization, serving configuration, GPU selection, batching strategies, cold start mitigation, adapter swapping.
• Optimize for spiky traffic (meetings end in 30-minute blocks) focusing on throughput curves.
• Build fine-tuning pipelines for repeatable infrastructure: from JSONL dataset to optimized model ready for serving (LoRA/QLoRA SFT, distillation, DPO).
• Benchmark quantization across GPU families, evaluate serving frameworks (vLLM vs SGLang).
• Optimize GPU spend and debug production inference issues.

Requirements

Hard Skills:

• Deep experience with LLM serving frameworks (vLLM, SGLang, TensorRT-LLM): tuning attention backends, scheduling, CUDA graph warmup, prefix caching.
• Hands-on quantization: weight vs activation, per-channel vs per-tensor scaling, dynamic vs static.
• Production fine-tuning: LoRA/QLoRA SFT, training frameworks (ms-swift, Axolotl, torchtune), data formatting, learning rate schedules.
• Strong Python for serving infrastructure, benchmarking, training pipelines.
• GPU profiling and performance analysis (compute, memory bandwidth, scheduling).

Strong Signals:

• Cost modeling for GPU infrastructure.
• Multimodal models (audio/vision + LLM).
• Modal, Ray Serve, serverless GPU platforms.
• Audio processing (codecs, chunking, sample rates).
• Building internal tooling.