# Software Engineer - Data Platform

**Company:** [xAI](https://hotfix.jobs/companies/xai)
**Location:** Palo Alto, CA, San Francisco, CA
**Role:** Data Engineering
**Salary:** $180k – $440k/yr
**Skills:** Kafka, Spark, Flink, Trino, Hdfs, Rust, Go, Scala, Distributed Systems, Stream Processing
**Posted:** 2026-03-06

> Builds and operates petabyte-scale data platform infrastructure using Kafka, Spark, Flink, and Trino to power real-time ML pipelines and analytics. Requires expertise in distributed systems, stream processing, and systems languages like Rust, Go, or Scala.

## Job Description

## About the Team
The Data Platform team builds and operates infrastructure for large-scale data transport and processing, managing Apache Kafka, HDFS, Spark, Flink, and Trino for real-time ML pipelines, feed ranking, experimentation, analytics, and observability at petabyte scale.

## About the Role
Design, build, and operate distributed systems powering data movement and compute, processing trillions of events daily for scalability, performance, and reliability in product and ML workloads.

## What You Will Do
- Design and implement high-throughput, low-latency data ingestion and transport systems.
- Scale and optimize multi-tenant Kafka infrastructure supporting real-time workloads.
- Extend and tune Spark, Flink, and Trino for demanding production pipelines.
- Build interfaces, APIs, and pipelines enabling teams to query, process, and move data at petabyte scale.
- Debug and optimize distributed systems, with a focus on reliability and performance under load.
- Collaborate with ML, product, and infrastructure teams to unblock critical data workflows.

## Ideal Candidate
- Proven expertise in distributed systems, stream processing, or large-scale data platforms.
- Proficiency in **Rust**, **Go**, **Scala** or similar systems languages.
- Hands-on experience with **Kafka**, **Flink**, **Spark**, **Trino**, or **Hadoop** in production.
- Strong debugging, profiling, and performance optimization skills.
- Track record of shipping and maintaining critical infrastructure.
- Comfortable working in fast-moving, high-stakes environments with minimal guardrails.

## Compensation and Benefits
**Annual Salary Range**: $180,000 - $440,000 USD  
Base salary plus equity, comprehensive medical, vision, dental, 401(k), disability insurance, life insurance, and perks.

## Similar roles

- [Data Engineer](https://hotfix.jobs/jobs/6b412eb4-37c6-4c36-8f77-10e86bc79c93) - Actively AI - New York, NY - $180k – $210k/yr
- [Data Engineer](https://hotfix.jobs/jobs/7443d425-2ed8-49a3-9b59-ee77171cd7d0) - Baseten - San Francisco, CA - $180k – $250k/yr
- [Software Engineer, Distributed Data Systems](https://hotfix.jobs/jobs/515ee00c-9aa9-4052-8be3-d18af7f2ea6a) - Exa - San Francisco, CA - $180k – $350k/yr
- [Software Engineer, Data Infrastructure](https://hotfix.jobs/jobs/cfc2b08d-e49a-4d20-a3f4-2f4fffbd8b88) - Datology AI - Redwood City, CA - $180k – $300k/yr
- [Data Engineer](https://hotfix.jobs/jobs/d5dae85f-a730-436c-a9e7-adb1ca41470d) - Airtable - San Francisco, CA - $180k – $222k/yr

**Apply:** https://hotfix.jobs/jobs/f6588639-59ab-484a-9fab-418122ada203
**Canonical:** https://hotfix.jobs/jobs/f6588639-59ab-484a-9fab-418122ada203