# Senior Software Engineer, Data Infrastructure

**Company:** [Arena](https://hotfix.jobs/companies/arena)
**Location:** San Francisco, CA, California
**Role:** Data Engineering
**Experience:** 5+ years
**Skills:** SQL, Python, Spark, Ray Data, Databricks, Snowflake, Delta Lake, Scala, R
**Posted:** 2025-12-18

> Build and scale data pipelines to process millions of user votes for AI model evaluation. Partner with researchers to deliver insights via dashboards, ensuring data quality and reliability in a fast-paced environment. Requires 5+ years in data engineering with big data tools.

## Job Description

## Responsibilities

- Design and build robust data pipelines to ingest, process, and transform user vote data to features essential for model performance evaluation.
- Collaborate with researchers and product leadership to understand product goals and necessary data.
- Design and implement solutions to generate result dashboards and reports, providing useful information for the public, model providers, and researchers.
- Ensure the integrity, data quality, and reliability of the pipelines.
- Scale our data infrastructure to accommodate increasing data volumes and evolving analytical needs.

## Requirements

- 5+ years of experience in software engineering, with a dedicated focus on data engineering and big data technologies.
- Proficiency in **SQL** and at least one programming language commonly used for data analysis (**Python** (preferred), Scala, R).
- Hands-on experience with data processing and pipeline frameworks (**Apache Spark**, **Ray Data**, etc.) and at least one popular big data analytics platform (**Databricks**, **Snowflake**).
- Demonstrated experience in designing, implementing, optimizing, and debugging production data pipelines.

## Preferred Qualifications

- Prior work in data analytics or datalake platforms.
- Experience in advanced data analysis tools, such as **Delta Lake**, streaming tables.
- Exposure to machine learning is a plus.

## Similar roles

- [Data Operations Lead](https://hotfix.jobs/jobs/4f8dee72-5fff-4c10-90b5-cbd0607417e6) - HappyRobot - San Francisco, CA - $180k – $240k/yr
- [Senior Software Engineer, AI Data Systems & Database Infrastructure](https://hotfix.jobs/jobs/11739793-220f-4ff1-882c-90bf341ca36c) - Ambient.ai - Redwood City, CA - $168k – $205k/yr
- [Senior Data Engineer](https://hotfix.jobs/jobs/cf45e5a5-fb7b-43fb-94e6-dff187027a77) - Vanta - Remote - $190k – $224k/yr
- [Senior Data Engineer](https://hotfix.jobs/jobs/c950f79a-6886-4ce0-9990-2627edb264f3) - Cloudflare - Austin, TX
- [Senior Data Engineer](https://hotfix.jobs/jobs/298b05d6-eba7-413d-bb1d-948ca80c669a) - Trexquant - Stamford, CT - $150k – $200k/yr

**Apply:** https://hotfix.jobs/jobs/d3fe6f3c-458f-46f3-8090-d0e4d80fe3d0
**Canonical:** https://hotfix.jobs/jobs/d3fe6f3c-458f-46f3-8090-d0e4d80fe3d0