Software Engineer

Responsibilities

• Own Baseten’s GPU fabric observability and root-cause analysis architecture.
• Build telemetry pipelines across switches, NICs, hosts, GPUs, Kubernetes, and inference services.
• Model topology, flow paths, service ownership, and failure domains.
• Separate true fabric faults from host, NIC, GPU, kernel, driver, RDMA, scheduler, and workload failures.
• Create clear operator workflows for triage, remediation, and post-incident learning.

Example Initiatives

• Real-time telemetry engine — Build the ingestion, reduction, storage, and query path for high-cardinality fabric, host, GPU, and workload telemetry.
• Service-aware fabric diagnosis — Build collectors, probes, and topology-aware correlation to detect latency, drops, stalls, congestion, bad paths, and degradation.
• Software-aware RDMA diagnosis — Tie network behavior to RDMA operations, GPUDirect paths, KV cache transfers, prefill/decode disaggregation, and request latency.

Requirements

• Staff-level or senior staff-level experience building production infrastructure software.
• Strong distributed systems background, especially streaming systems, telemetry pipelines, diagnostics, or control-plane software.
• Experience building systems that process high-volume, high-cardinality, noisy operational data.
• Understanding of networking fundamentals and high-performance networks.
• Ability to work with low-level infrastructure signals and build practical correlation, anomaly detection, or root-cause analysis systems.

Nice-to-Haves

• Experience with RDMA data paths, GPUDirect transfers, prefill/decode disaggregation, KV cache movement, request routing, and workload backpressure.
• Familiarity with RoCE, InfiniBand, ECN/DCQCN, RNIC, GPU drivers, and related technologies.

Benefits

• Competitive compensation, including meaningful equity.
• 100% coverage of medical, dental, and vision insurance for employee and dependents.
• Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!).
• Paid parental leave.
• Fertility and family-building stipend through Carrot.
• Company-facilitated 401(k).
• Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.