Machine-Learning Operations Engineer

Responsibilities

• Architect, design, and implement ML software systems for deploying and managing models at scale.
• Develop and maintain infrastructure that supports efficient ML operations, including data pipelines, model evaluations, deployments, and training at scale.
• Collaborate closely with ML engineers, software engineers, and cross-functional teams to ensure seamless integration of models with data pipelines and products.
• Troubleshoot production issues and continuously improve systems to enhance performance and efficiency.
• Create tooling for online and offline evaluation of ML & LLM systems.

Requirements

• 5+ years of experience in ML model deployment, infrastructure, and scaling in production environments.
• Strong software engineering fundamentals, with proficiency in Python and TypeScript.
• Experience in software design and architecture for highly available ML systems for use cases like inference, evaluation, and experimentation.
• Strong knowledge of observability, including logging, metrics, tracing, model performance monitoring, and alerting.
• Experience with distributed systems, reliability, and production incident response.
• Comfortable working in ambiguity with high ownership, moving quickly in a fast-paced startup environment, and proactively driving projects from idea to production.

Nice-to-Haves

• Experience working with ML CI/CD and common ML frameworks like PyTorch, TensorFlow, etc.
• Experience working with common inference frameworks like vLLM, TensorRT, Triton, etc.
• Experience with GPU orchestration, including managing GPU workloads/scheduling, cost management, cluster utilization, etc.
• Experience with GPU optimization (training/inference) involving CUDA profiling, memory optimization, multi-GPU communication, etc.