Senior Technical Program Manager

Responsibilities

• Own end-to-end delivery of AI infrastructure programs—from model training pipelines and experiment tracking to inference serving and production monitoring.
• Define technical architecture, integration patterns, and rollout strategies for new ML systems and tooling (e.g., vector databases, model servers, evaluation frameworks, prompt engineering platforms).
• Serve as connective tissue between ML research, ML engineering, product, and data teams to align on ML system requirements, capability roadmaps, and deployment timelines.
• Drive cost and latency optimization for real-time inference workloads at scale.
• Build lightweight internal tools and processes to accelerate ML iteration cycles (experiment tracking, model versioning, A/B testing infrastructure).
• Identify and resolve technical bottlenecks in training pipelines, serving infrastructure, and model evaluation workflows.
• Work closely with ML practitioners to translate research breakthroughs into scalable, observable systems.

Requirements

• 5+ years of program management or technical leadership in ML infrastructure, ML platforms, or AI tooling (or equivalent).
• Strong technical acumen in ML systems—ideally hands-on experience as an ML engineer, systems engineer, or ML infrastructure engineer.
• Experience coordinating cross-functional ML programs (e.g., model training → evaluation → serving → monitoring).
• Proven ability to translate ML/research requirements into robust, scalable infrastructure.
• Comfortable working in ambiguity and helping teams navigate complex technical tradeoffs (e.g., accuracy vs. latency vs. cost).
• Excellent communication with both technical and non-technical stakeholders.
• Familiarity with high-growth or startup environments.

Nice-to-Haves

• Hands-on experience with model serving frameworks (vLLM, TensorRT, TorchServe, or similar).
• Experience optimizing LLM or speech/audio model inference (quantization, distillation, KV-cache optimization, batching strategies).
• Familiarity with ML experiment tracking and versioning tools (MLflow, Weights & Biases, DVC, or similar).
• Background in feature stores, vector databases, or real-time ML systems.
• Knowledge of cost optimization for GPU/ML workloads on cloud and on-premise infrastructure.
• Experience with multi-region model serving or edge deployment.
• Hands-on with relevant frameworks (PyTorch, CUDA, Hugging Face, etc.) or cloud platforms (AWS SageMaker, GCP Vertex AI, Azure ML).