# Senior Infra Engineer: Observability

**Company:** [Railway](https://hotfix.jobs/companies/railway)
**Location:** Remote
**Role:** DevOps / SRE
**Skills:** Go, Rust, gRPC, Terraform, Ansible, GraphQL, TypeScript, Victoriametrics, ClickHouse, Distributed Systems
**Posted:** 2026-05-01

> Build high-scale observability pipelines and alerting engines handling 1M+ RPS for logs/metrics, develop Golang/Rust gRPC services and APIs, and manage immutable infrastructure with Terraform/Ansible in a distributed systems environment.

## Job Description

## Responsibilities
- Build ingestion pipelines to consume 1M+ RPS streams of logs, metrics, and other telemetry
- Build scalable, fault tolerant alerting engines for notifying users, in real-time, of threshold breaches
- Craft rich backend observability APIs, working with product to build amazing experiences for instantly grokking their application
- Provide APIs to access realtime log/metrics streams to be consumed by the Dashboard and Product Teams
- Build **Golang**/**Rust** GRPC services from scratch capable of supporting tens of thousands of users
- Define infrastructure that can be torn down, failed over, and reconstituted from scratch using principle of immutable infrastructure using **Terraform** and **Ansible**
- Write Engineering Requirement Documents to take something from idea, to defined tasks, to implementation, to monitoring it’s success
- Interface with our **TypeScript** and **GraphQL** edge to expose your microservice APIs for both internal and potentially external consumption

## Requirements
- Strong understanding of **distributed systems**
- Enjoy building fault tolerant, resilient, and scalable services
- Interests in **VictoriaMetrics**, **ClickHouse**, and other systems for building observability stacks from the ground up
- Solid intuition about how long your solutions will last
- Implement solutions, create monitors for error boundaries, and document requirements
- Great sense of direction and prioritization when dealing with ambiguity of an early stage startup
- Sense of grit to dive into a problem, implement a solution, scale that solution, and replace it when needed
- Great communication skills

## Similar roles

- [Site Reliability Engineer](https://hotfix.jobs/jobs/ab64f10b-7c19-4c49-b827-0e3559c514c9) - Forward Networks - Santa Clara, CA - $230k – $250k/yr
- [Sr. Software Engineer, Platform Engineering](https://hotfix.jobs/jobs/fafdb79e-697e-4378-aa19-4482d0884fe1) - Chronograph - Remote - $175k – $215k/yr
- [Senior Site Reliability Engineer](https://hotfix.jobs/jobs/076df34d-0d8e-4f15-8a79-eb559a8e5f52) - Talkiatry - Remote - $160k – $185k/yr
- [Senior Infrastructure Engineer](https://hotfix.jobs/jobs/7fad3112-9910-402c-b20e-17b838f39c55) - Aurelian - Seattle, WA - $160k – $220k/yr
- [Sr. Site Reliability Engineer](https://hotfix.jobs/jobs/2c837b9b-f230-4e2a-aadb-21d12039b4d2) - Illumio - Sunnyvale, CA - $170k – $196k/yr

**Apply:** https://hotfix.jobs/jobs/8a7ad0ac-6711-4d8d-9df0-833f2d1bfb85
**Canonical:** https://hotfix.jobs/jobs/8a7ad0ac-6711-4d8d-9df0-833f2d1bfb85