Skip to content
CohereCohereSan Francisco, CA

Lead Member of Technical Staff, Inference Infrastructure

Leads architecture and strategy for deploying optimized NLP models in high-throughput, low-latency production environments using Kubernetes and cloud platforms. Mentors engineers and designs custom customer deployments with 8+ years infrastructure experience.

Salary not listed
Remote8+ YOEDevOps / SRE

About the role

Responsibilities

  • Provide technical leadership across multiple teams, driving architecture and strategy for deploying optimized NLP models to production in low latency, high throughput, high availability environments.
  • Serve as key point of contact for customers, leading design of customized deployments.
  • Mentor engineers to raise technical bar.

Requirements

  • 8+ years engineering experience running production infrastructure at large scale, with technical leadership track record.
  • Experience leading architecture/design of large, highly available distributed systems with Kubernetes and GPU workloads.
  • Deep expertise with Kubernetes dev/production coding/support, setting team-wide standards.
  • Extensive experience across GCP, Azure, AWS, OCI, multi-cloud/on-prem/hybrid environments.
  • Lead design, deployment, support, troubleshooting of complex Linux-based computing environments at scale.
  • Own compute/storage/network resource and cost management at organizational level.
  • Expertise in computational characteristics of accelerators (GPUs, TPUs, custom), leveraging for latency/throughput improvements.
  • Deep knowledge of distributed systems, establishing patterns/practices.
  • Proficiency in Golang, C++ or similar for high-performance scalable servers.

Nice-to-Haves

  • Exceptional collaboration, communication, mentoring, cross-functional leadership.
  • Grit and adaptability for complex technical challenges.

Skills

KubernetesGCPAzureAWSOciGoC++LinuxGpusTpusDistributed Systems

Similar roles

DevOps / SRE jobs
Crusoe

Staff Infrastructure Engineer

CrusoeSan Francisco, CA +1

Staff Cloud Infrastructure Engineer responsible for managing Crusoe’s cloud fleet operations, building IaC automation for bare-metal server provisioning/reprovisioning, scaling deployments, GPU hardware troubleshooting, and transitioning to Kubernetes/containerized workflows.

208k – 253k/yr
On-site7+ YOEDevOps / SRE
Aurelian

Staff Infrastructure Engineer

AurelianSeattle, WA

Staff Infrastructure Engineer building analytics, observability, and developer tooling for Aurelian's real-time AI agents that support 911 emergency call centers. Requires 6+ years in infrastructure/platform/backend roles with experience in reliability and scale.

200k – 300k/yr
On-site6+ YOEDevOps / SRE
Stripe

Staff Engineer, Deployment Platform

StripeSeattle, WA

Staff Engineer owning end-to-end delivery of Stripe's deployment platform, including orchestrator evolution, Kubernetes fleet migration, anomaly detection, and reliability. Requires 10+ years experience leading large-scale infrastructure projects with deep expertise in distributed systems, containers, and operational excellence.

Salary not listed
Hybrid10+ YOEDevOps / SRE
Crusoe

Staff Production Engineer, Compute

CrusoeSan Francisco, CA

Staff Production Engineer responsible for developing automation/observability, scaling virtualization (KVM/QEMU), optimizing Linux kernel performance, and supporting AI/HPC workloads on CPU/GPU/DPU hardware. Requires 8+ years in Linux systems engineering, kernel internals, and virtualization.

209k – 253k/yr
On-site8+ YOEDevOps / SRE
Basis

Member of Technical Staff

BasisNew York, NY

Site Reliability Engineer building and operating scalable, reliable infrastructure for an AI-powered accounting platform at a fast-growing startup. Requires 5+ years production infrastructure experience, strong software engineering skills, IaC, observability, and on-call ownership in an onsite NYC role.

100k – 300k/yr
On-site5+ YOEDevOps / SRE