Software Engineer, Collective Communication
Designs and implements custom networking collectives using C++ and CUDA for efficient AI model training on supercomputers. Requires expertise in low-level performance-critical software, RDMA distributed algorithms, and GPU/CPU code.
About the job
Responsibilities
- Collaborate closely with ML researchers to design and implement efficient collective operations in C++ and CUDA.
- Ensure that our largest training jobs take full advantage of the different network transports used in our supercomputers.
- Work on simulations to inform our future supercomputer network designs.
Requirements
- Background in low level performance critical software.
- Have written distributed algorithms using RDMA in the past.
- Comfortable writing low level performance sensitive CPU and/or GPU code.
- Familiar with network simulation techniques.
Nice-to-haves
- Experience with collective communication.
Skills
C++, CUDA, Rdma, Distributed Algorithms, Network Simulation, Collective Communication, Gpu Programming, Network Transports
Similar jobs
Backend Engineering jobs
Anthropic
San Francisco, CABuild and scale reliable research infrastructure and distributed systems for evolving AI research workflows. The role independently leads complex technical projects, makes foundational architectural decisions, and partners with researchers and engineering teams.
Thinking Machines Lab
San Francisco, CA
Build and operate secure, scalable sandboxing infrastructure for untrusted, model-generated code and tool calls. The role requires backend programming, virtualization or isolation experience, and strong knowledge of Linux security primitives.
OpenAI
San Francisco, CA
Build backend systems, data infrastructure, agentic workflows, and enterprise integrations that make AI useful and trustworthy for financial institutions. The role requires 5+ years of software engineering experience, backend-language proficiency, and experience with production data-intensive or distributed systems.
OpenAI
San Francisco, CABuild and operate Host Assurance services and host software that establish trust in bare-metal and virtual machine infrastructure through secure bootstrap, identity, attestation, and verification. The role requires production software engineering experience across reliable systems, platform or infrastructure security, and host-system boundaries.
OpenAI
San Francisco, CABuild backend infrastructure and customer-facing workflows that enable developers and AI agents to operate reliably in secure cloud environments. The role requires strong Go and production backend experience, distributed-systems expertise, and practical knowledge of cloud infrastructure, networking, and security.