# Staff Data Engineer

**Company:** [Twenty](https://hotfix.jobs/companies/twenty)
**Location:** New York, NY
**Role:** Data Engineering
**Experience:** 8+ years
**Skills:** Apache Iceberg, Delta Lake, Apache Hive, Trino, Presto, Aws Athena, Spark, ClickHouse, Amazon Redshift, Google Bigquery, Airflow, Aws Glue, Kafka, RabbitMQ, Neo4J
**Posted:** 2026-05-19

> Lead data infrastructure for a national security tech company building a high-performance data lake and ETL pipelines for petabyte-scale cyber operations datasets. Requires 8+ years experience, strong data lake expertise, and proven technical leadership.

## Job Description

## Responsibilities
- Lead the development and operation of a data lake for cyber operations and intelligence data
- Design schemas, partitions, and indexes that make complex datasets performant and cost-effective to query
- Partner with engineers and intelligence analysts to define query patterns and data products for mission use cases
- Build and evolve ETL pipelines that are observable, recoverable, and resilient to upstream change
- Drive technical initiatives end-to-end, from architecture decisions through production rollout and iteration
- Establish best practices for data quality, documentation, and operational ownership across the platform
- Mentor engineers on data modeling, performance tuning, and production-grade pipeline design
- Identify bottlenecks in storage/compute/query layers and ship improvements with clear performance wins

## Requirements
- 8+ years of experience in data engineering and/or data architecture
- Mastery-level expertise building ETL pipelines and operating them in production
- Deep experience with data lake architecture and systems used to query data lakes
- Strong schema and index design skills, including partitioning, indexing, and clustering strategies
- Experience with column-oriented databases in production environments
- Built data systems from scratch (not only maintained existing platforms)
- Proven leadership experience mentoring engineers and driving technical initiatives
- U.S. citizen and able to meet the role’s security requirements

## Nice to Have
- Experience with key-value datastores
- Worked with streaming and message queue systems
- Experience with graph database technologies
- Worked with internet/networking datasets (e.g., scan data, DNS, netflow, certificates)
- Experience supporting analysts or operational users with high-stakes data needs

## Tech Environment
- Data lakes: Apache Iceberg, Delta Lake, Apache Hive
- Query engines: Trino, Presto, AWS Athena, Apache Spark
- Column stores: ClickHouse, Amazon Redshift, Google BigQuery
- ETL / orchestration: Airflow, AWS Glue, NiFi, ClickPipe
- Streaming / queues: Kafka, RabbitMQ, NATS, AWS Kinesis
- Graph: Neo4j, AWS Neptune, Memgraph, Apache AGE

## Similar roles

- [Staff+ Software Engineer, Capacity Engineering](https://hotfix.jobs/jobs/0649082b-f46e-44be-97b6-e5f7f474e36a) - Anthropic - San Francisco, CA - $320k – $485k/yr
- [Staff Software Engineer, Communication & Connectivity](https://hotfix.jobs/jobs/db2a925e-c86f-4b1c-959e-0319e66571e8) - Airbnb - Remote - $204k – $255k/yr
- [Staff Software Engineer](https://hotfix.jobs/jobs/91db0e3d-778f-43d0-8791-6211fd411ca4) - Rippling - Seattle, WA - $189k – $315k/yr
- [Staff Data Engineer](https://hotfix.jobs/jobs/043e5ff7-17b7-46b1-af3a-36c0084a4ac6) - Checkr - Denver, CO - $166k – $230k/yr
- [Staff+ Software Engineer, Databases](https://hotfix.jobs/jobs/810c29c6-a707-4585-8787-9b3a2a462e0d) - Anthropic - San Francisco, CA - $320k – $485k/yr

**Apply:** https://hotfix.jobs/jobs/d65846cd-5670-4150-be9e-a0a45e7045bb
**Canonical:** https://hotfix.jobs/jobs/d65846cd-5670-4150-be9e-a0a45e7045bb