Site Reliability Engineer (SRE)

What You’ll Do

• Define and own end-to-end reliability, from CI/CD flows to production observability and incident response.
• Develop appropriate Service Level Objectives for distributed training systems, balancing job completion reliability and scheduling latency with development velocity.
• Design and implement monitoring and observability across the full training path.
• Drive incident response for Tinker platform issues, ensuring rapid recovery, thorough incident reviews, and systematic improvements that prevent recurrence.
• Harden multi-tenant isolation and resource scheduling so that LoRA-based workload co-scheduling maximizes utilization without compromising reliability or data separation
• Collaborate with security teams to address production vulnerabilities

Skills and Qualifications

Minimum qualifications:

• Bachelor's degree or equivalent experience in computer science, engineering, or similar.
• Experience in distributed systems, cloud infrastructure, or site reliability engineering.
• Proficiency writing software to solve reliability problems, including building tooling and automation.
• Experience with production incident response, postmortems, and systematic reliability improvement.
• Strong communication skills and track record of coordination across engineering and research teams.

Preferred qualifications:

• Deep experience operating production cloud services at scale (e.g., public cloud platforms, internal cloud services)
• Background in distributed training frameworks and how infrastructure failures surface in training behavior.
• Track record building checkpoint and recovery systems for long-running distributed jobs.
• Expertise in Kubernetes at scale: deploying, operating, debugging, and tuning clusters handling heterogeneous GPU workloads.

Compensation

Expected annual salary range: $350,000 – $475,000 USD.

Benefits

Generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.