# Site Reliability Engineer, Inference Infrastructure

**Company:** [Cohere](https://hotfix.jobs/companies/cohere)
**Location:** Remote
**Role:** DevOps / SRE
**Experience:** 5+ years
**Skills:** Kubernetes, Gpu Workloads, GCP, Azure, AWS, Oracle Cloud Infrastructure, Linux, Distributed Systems, Go, C++, Observability, SLOs, On-Call Operations, Nlp Models, Multi-Cloud
**Posted:** 2026-01-12

> Operates and scales reliable infrastructure for serving Cohere’s language models, building Kubernetes automation, observability, resilience, and customized production deployments. Requires 5+ years running large-scale production infrastructure and experience with distributed systems, cloud platforms, Linux, GPU workloads, and high-performance server development.

## Job Description

## Responsibilities
- Build self-service systems that automate managing, deploying, and operating services, including custom Kubernetes operators supporting language model deployments.
- Automate environment observability and resilience so developers can troubleshoot and resolve problems.
- Help ensure defined SLOs are met, including participation in an on-call rotation.
- Build strong relationships with internal developers and influence the Infrastructure team’s roadmap based on their feedback.
- Develop the team through knowledge sharing and an active review process.
- Deploy optimized NLP models to production in low-latency, high-throughput, and high-availability environments.
- Interface with customers and create customized deployments for their specific needs.

## Requirements
- 5+ years of engineering experience running production infrastructure at a large scale.
- Experience designing large, highly available distributed systems with Kubernetes and GPU workloads on those clusters.
- Experience with Kubernetes development, production coding, and support.
- Experience with GCP, Azure, AWS, OCI, and multi-cloud, on-premises, or hybrid serving.
- Experience designing, deploying, supporting, and troubleshooting complex Linux-based computing environments.
- Experience with compute, storage, network resource, and cost management.
- Strong collaboration and troubleshooting skills for building mission-critical systems and ensuring smooth operations.
- Grit and adaptability to solve evolving complex technical challenges.
- Familiarity with computational characteristics of GPUs, TPUs, and/or custom accelerators, especially their influence on inference latency and throughput.
- Strong understanding of or working experience with distributed systems.
- Experience with Golang, C++, or other languages designed for high-performance, scalable servers.

## Benefits and Compensation
- Weekly lunch stipend of $75/£75 or equivalent in local currency.
- Full health and dental benefits, including a separate mental health budget.
- RRSP matching, 401(k), or pension scheme.
- 100% parental leave top-up for up to 6 months for either parent.
- Annual enrichment benefits for arts and culture, fitness and wellness, quality time, and workspace improvements.
- Education and learning stipend for conferences, courses, and coaching.
- Six weeks of paid vacation (30 working days).
- Travel budget for remote employees to visit other offices and an annual company offsite.
- Coworking benefit for employees not near an office.
- $500 home office stipend.

## Similar jobs

- [Software Engineer: Resiliency - Deploy at Scale](https://hotfix.jobs/jobs/21bb6f4f-daea-46ab-bac8-fff083d17451) - Cloudflare - London, United Kingdom
- [Software Engineer, Infrastructure](https://hotfix.jobs/jobs/585da47a-e02d-4c63-9816-248a2faa9b5b) - Granica - Remote
- [Capacity Ops Engineer](https://hotfix.jobs/jobs/f1904714-7dd3-4ee3-9e7a-e4fcf52083bd) - Baseten - San Francisco, CA - $170k – $230k/yr
- [IT Security and Automation Engineer](https://hotfix.jobs/jobs/604b87b5-13a2-4bba-88b2-f7d0fbbad141) - Teleport - Remote - $149k – $258k/yr
- [Electrical Field Engineer - Data Center](https://hotfix.jobs/jobs/6bfa0e4c-9ccf-438a-b65e-cd4c6297762c) - Crusoe - Remote - $196k – $235k/yr

**Apply:** https://hotfix.jobs/jobs/2459cc99-8e99-4c75-8530-28ec0fd1d92c
**Canonical:** https://hotfix.jobs/jobs/2459cc99-8e99-4c75-8530-28ec0fd1d92c