# Data Engineer, Analytics

**Company:** [OpenAI](https://hotfix.jobs/companies/openai)
**Location:** San Francisco, CA
**Role:** Data Engineering
**Salary:** $230k – $385k/yr
**Experience:** 3+ years
**Skills:** Python, Scala, Java, Spark, Airflow, Hadoop, Flink, S3, Hdfs, Dagster
**Posted:** 2023-09-26

> Build and manage data pipelines and canonical datasets for product metrics, safety systems, and business decisions. Collaborate with cross-functional teams including Data Science and Research; requires 3+ years data engineering experience with Spark, ETL tools, and distributed systems.

## Job Description

## Responsibilities
- Design, build and manage our data pipelines, ensuring all user event data is seamlessly integrated into our data warehouse.
- Develop canonical datasets to track key product metrics including user growth, engagement, and revenue.
- Work collaboratively with various teams, including Infrastructure, Data Science, Product, Marketing, Finance, and Research to understand their data needs and provide solutions.
- Implement robust and fault-tolerant systems for data ingestion and processing.
- Participate in data architecture and engineering decisions, bringing your strong experience and knowledge to bear.
- Ensure the security, integrity, and compliance of data according to industry and company standards.

## Requirements
- 3+ years of experience as a data engineer and 8+ years of any software engineering experience (including data engineering).
- Proficiency in at least one programming language commonly used within Data Engineering, such as **Python**, **Scala**, or **Java**.
- Experience with distributed processing technologies and frameworks, such as **Hadoop**, **Flink** and distributed storage systems (e.g., **HDFS**, **S3**).
- Expertise with any of ETL schedulers such as **Airflow**, **Dagster**, **Prefect** or similar frameworks.
- Solid understanding of **Spark** and ability to write, debug and optimize Spark code.

## Similar roles

- [Software Engineer, Distributed Data Systems (Sora)](https://hotfix.jobs/jobs/c9a6b124-c7e4-4e61-859c-b27e265a9146) - OpenAI - San Francisco, CA - $230k – $385k/yr
- [Software Engineer (Ray Data)](https://hotfix.jobs/jobs/947b6d4c-ec41-4f52-949c-262bc566f755) - Anyscale - San Francisco, CA - $226k – $241k/yr
- [Software Engineer, Data Infrastructure](https://hotfix.jobs/jobs/a8009f18-5785-47c4-a091-169beaee14fd) - The Voleon Group - Remote - $235k – $300k/yr
- [Data Engineering Manager, Data & ML Platform](https://hotfix.jobs/jobs/69306472-f039-43db-b785-2e0261f96a70) - Hinge Health - San Francisco, CA - $220k – $330k/yr
- [Software Engineer, Data Infrastructure - Research](https://hotfix.jobs/jobs/c7766506-4120-420e-b7ac-08ad093646dc) - OpenAI - San Francisco, CA - $250k – $380k/yr

**Apply:** https://hotfix.jobs/jobs/8438cfd5-3c2e-4dfb-a809-bc71da60b0db
**Canonical:** https://hotfix.jobs/jobs/8438cfd5-3c2e-4dfb-a809-bc71da60b0db