# Software Engineer — GPU Networking & Distributed Systems

**Company:** [Baseten](https://hotfix.jobs/companies/baseten)
**Location:** San Francisco, CA, California
**Role:** Backend Engineering
**Salary:** $150k – $250k/yr
**Skills:** Rdma, Roce, InfiniBand, Nccl, Nvshmem, Ucx, C++, Python, Tensorrt-Llm, Nvlink
**Posted:** 2026-02-23

> Software engineer architecting GPU networking software with RDMA for distributed AI inference. Optimizes communication for H100/Blackwell clusters, WideEP, and serverless LLM startups using C++/Python and NVIDIA tools.

## Job Description

## What You'll Do
- **Make RDMA First-Class**: Integrate RDMA/RoCE/InfiniBand into inference stack for bandwidth/latency improvements.
- **Optimize Distributed Inference**: Implement networking for Disaggregated KV Cache Offload and WideEP across NVLink/InfiniBand.
- **Enable Serverless-Grade Startup Speeds**: Work on checkpointing/storage for sub-10s startup of trillion-parameter models.
- **Deep-Dive into Hardware**: Characterize H100/H200, B200/B300, GB200/300 NVL72 clusters with acceptance tests.
- **Build Observability**: Design tools for packet flow, congestion, bandwidth visualization.
- **Optimize Kernels**: Use NCCL, NVSHMEM; write custom communication kernels.

## Who You Are
- Deep experience with high-performance networking (InfiniBand, RoCE v2).
- Fluent in **C++** or **Python**, understanding NVIDIA memory hierarchy (H100/Blackwell).
- Comfortable diving into TensorRT-LLM source, custom bindings, NVLink debugging.
- Know when to use/build custom solutions beyond standard tools.

## Highly Preferred
- Deep knowledge of **NCCL**, **NVSHMEM**, **UCX**.
- Experience with **GPUDirect Storage (GDS)**, Weka/3FS.
- Familiarity with **TensorRT-LLM**, **vLLM**, **Sglang**.
- Running low-level benchmarks for hardware qualification.

## Benefits
- Competitive compensation with equity.
- 100% medical/dental/vision coverage.
- Generous PTO, Winter Break.
- Paid parental leave, 401(k).

## Similar roles

- [Software Engineer, Workers Deploy & Config](https://hotfix.jobs/jobs/373e862c-92c6-4938-907d-a09a407f2835) - Cloudflare - New York, NY - $150k – $206k/yr
- [Backend Software Engineer](https://hotfix.jobs/jobs/08928e42-7bb9-4744-9124-acc9cde7ae0e) - Doxel - San Francisco, CA - $150k – $200k/yr
- [Deliverability Engineer, Core Sending](https://hotfix.jobs/jobs/0d8528d0-1960-4a62-867c-c7a9d2b9e09a) - Resend - Remote - $150k – $180k/yr
- [Backend Engineer , Core Sending](https://hotfix.jobs/jobs/158a2892-6b94-4186-ae37-b322d0659fca) - Resend - Remote - $150k – $170k/yr
- [Software Engineer — Integrations](https://hotfix.jobs/jobs/13c07401-ac9b-4d48-8c31-8fc220a67151) - Kodex - Remote - $150k – $180k/yr

**Apply:** https://hotfix.jobs/jobs/1e7220c5-3159-4fcd-b481-0ecde1a74cba
**Canonical:** https://hotfix.jobs/jobs/1e7220c5-3159-4fcd-b481-0ecde1a74cba