# Lead Member of Technical Staff, Inference Infrastructure

**Company:** [Cohere](https://hotfix.jobs/companies/cohere)
**Location:** Remote
**Role:** DevOps / SRE
**Experience:** 8+ years
**Skills:** Kubernetes, GCP, Azure, AWS, Oci, Go, C++, Linux, Gpus, Tpus, Distributed Systems
**Posted:** 2026-04-28

> Leads architecture and strategy for deploying optimized NLP models in high-throughput, low-latency production environments using Kubernetes and cloud platforms. Mentors engineers and designs custom customer deployments with 8+ years infrastructure experience.

## Job Description

## Responsibilities
- Provide technical leadership across multiple teams, driving architecture and strategy for deploying optimized NLP models to production in low latency, high throughput, high availability environments.
- Serve as key point of contact for customers, leading design of customized deployments.
- Mentor engineers to raise technical bar.

## Requirements
- 8+ years engineering experience running production infrastructure at large scale, with technical leadership track record.
- Experience leading architecture/design of large, highly available distributed systems with **Kubernetes** and GPU workloads.
- Deep expertise with **Kubernetes** dev/production coding/support, setting team-wide standards.
- Extensive experience across **GCP**, **Azure**, **AWS**, **OCI**, multi-cloud/on-prem/hybrid environments.
- Lead design, deployment, support, troubleshooting of complex Linux-based computing environments at scale.
- Own compute/storage/network resource and cost management at organizational level.
- Expertise in computational characteristics of accelerators (**GPUs**, **TPUs**, custom), leveraging for latency/throughput improvements.
- Deep knowledge of distributed systems, establishing patterns/practices.
- Proficiency in **Golang**, **C++** or similar for high-performance scalable servers.

## Nice-to-Haves
- Exceptional collaboration, communication, mentoring, cross-functional leadership.
- Grit and adaptability for complex technical challenges.

## Similar roles

- [Staff Infrastructure Engineer](https://hotfix.jobs/jobs/bd50f619-96c5-4ab7-b28e-a3646ed6c4a6) - Crusoe - San Francisco, CA - $208k – $253k/yr
- [Staff Infrastructure Engineer](https://hotfix.jobs/jobs/75f07283-34de-4989-b193-11c3c3676fd2) - Aurelian - Seattle, WA - $200k – $300k/yr
- [Staff Engineer, Deployment Platform](https://hotfix.jobs/jobs/93017497-72c8-4479-ae04-8852d7c9f086) - Stripe - Seattle, WA
- [Staff Production Engineer, Compute](https://hotfix.jobs/jobs/7a9a870e-9394-42b9-b5f9-423d777df634) - Crusoe - San Francisco, CA - $209k – $253k/yr
- [Member of Technical Staff](https://hotfix.jobs/jobs/70236390-de41-405f-8319-26f7b1743953) - Basis - New York, NY - $100k – $300k/yr

**Apply:** https://hotfix.jobs/jobs/28810edf-3df4-4bfd-baf8-323af55a8726
**Canonical:** https://hotfix.jobs/jobs/28810edf-3df4-4bfd-baf8-323af55a8726