# Member of Technical Staff, Inference

**Company:** [Runway](https://hotfix.jobs/companies/runway)
**Location:** Remote
**Role:** ML Engineering
**Salary:** $240k – $290k/yr
**Experience:** 4+ years
**Skills:** PyTorch, Kubernetes, AWS, Terraform, Prometheus, Grafana, Torchscript, Flyte, Kueue, Kyverno
**Posted:** 2026-01-15

> ML infrastructure engineer productionizing generative AI models from research to scale, optimizing multi-GPU inference, and building reliable serving systems. Requires 4+ years ML inference experience, PyTorch, Kubernetes expertise.

## Job Description

## Responsibilities
- Productionize model checkpoints end-to-end: from research completion to internal testing to production deployment to post-release support
- Build and optimize inference systems for large-scale generative models running on multi-GPU environments
- Design and implement model serving infrastructure specialized for diffusion models and real-time diffusion workflows
- Add monitoring and observability for new model releases—track errors, throughput, GPU utilization, and latency
- Embed with research teams to gather training data, run preprocessing scripts, and support the model development process
- Explore and integrate with GPU inference providers (Modal, E2E, Baseten, etc.)

## Requirements
- 4+ years of experience running ML model inference at scale in production environments
- Strong experience with **PyTorch** and multi-GPU inference for large models
- Experience with **Kubernetes** for ML workloads—deploying, scaling, and debugging GPU-based services
- Comfortable working across multiple cloud providers and managing GPU driver compatibility
- Experience with monitoring and observability for ML systems (errors, throughput, GPU utilization)
- Self-starter who can work embedded with research teams and move fast
- Strong systems thinking and pragmatic approach to production reliability

## Nice to Haves
- Experience building custom inference frameworks or serving systems
- Deep understanding of distributed training and inference patterns (**FSDP**, data parallelism, tensor parallelism)
- Ability to debug low-level issues: NCCL networking problems, CUDA errors, memory leaks, performance bottlenecks
- Experience with diffusion models or video generation systems
- Knowledge of real-time or latency-sensitive ML applications

## Similar roles

- [Staff Software Engineer](https://hotfix.jobs/jobs/3ff71c05-9160-4c22-8a7d-cabd5b98e092) - Traba - New York, NY - $240k – $300k/yr
- [Staff Software Engineer](https://hotfix.jobs/jobs/60c76f48-09f3-413f-a9d8-8204704ba5fb) - Traba - New York, NY - $240k – $300k/yr
- [Staff Applied AI Engineer, AI Experiences & Agents](https://hotfix.jobs/jobs/95ea8b1a-3575-49ab-bca9-308add5c11b6) - Linktree - Los Angeles, CA - $240k – $250k/yr
- [Staff AI Engineer](https://hotfix.jobs/jobs/9dcf47a8-530d-4e8c-96a3-1831da2e82a8) - Fieldguide - San Francisco, CA - $240k – $310k/yr
- [Staff Software Engineer - Applied AI](https://hotfix.jobs/jobs/efd7b634-6bb8-48a4-bd45-aaceaf6fac38) - Ironclad - San Francisco, CA - $240k – $295k/yr

**Apply:** https://hotfix.jobs/jobs/f0d229c1-d1ca-454d-b0b5-77379f49e9df
**Canonical:** https://hotfix.jobs/jobs/f0d229c1-d1ca-454d-b0b5-77379f49e9df