Lead data infrastructure for a national security tech company building a high-performance data lake and ETL pipelines for petabyte-scale cyber operations datasets. Requires 8+ years experience, strong data lake expertise, and proven technical leadership.
Salary not listed
On-site8+ YOEData Engineering
About the role
Responsibilities
Lead the development and operation of a data lake for cyber operations and intelligence data
Design schemas, partitions, and indexes that make complex datasets performant and cost-effective to query
Partner with engineers and intelligence analysts to define query patterns and data products for mission use cases
Build and evolve ETL pipelines that are observable, recoverable, and resilient to upstream change
Drive technical initiatives end-to-end, from architecture decisions through production rollout and iteration
Establish best practices for data quality, documentation, and operational ownership across the platform
Mentor engineers on data modeling, performance tuning, and production-grade pipeline design
Identify bottlenecks in storage/compute/query layers and ship improvements with clear performance wins
Requirements
8+ years of experience in data engineering and/or data architecture
Mastery-level expertise building ETL pipelines and operating them in production
Deep experience with data lake architecture and systems used to query data lakes
Strong schema and index design skills, including partitioning, indexing, and clustering strategies
Experience with column-oriented databases in production environments
Built data systems from scratch (not only maintained existing platforms)
Proven leadership experience mentoring engineers and driving technical initiatives
U.S. citizen and able to meet the role’s security requirements
Nice to Have
Experience with key-value datastores
Worked with streaming and message queue systems
Experience with graph database technologies
Worked with internet/networking datasets (e.g., scan data, DNS, netflow, certificates)
Experience supporting analysts or operational users with high-stakes data needs
Tech Environment
Data lakes: Apache Iceberg, Delta Lake, Apache Hive
Build and operate production data pipelines, observability tools, and planning systems to maximize utilization, efficiency, and attribution of Anthropic's large-scale multi-cloud accelerator and CPU fleet. Requires strong Python/SQL, cloud operations, and Kubernetes experience in a high-ambiguity environment.
320k – 485k/yr
Hybrid7+ YOEData Engineering
Staff Software Engineer, Communication & Connectivity
AirbnbUnited States
Staff Software Engineer leading design and development of large-scale batch and real-time data pipelines and ML infrastructure to power GenAI/LLM products and features for Airbnb's Messaging, Notifications, and Connectivity organization. Requires 9+ years experience building production ML systems and cross-functional collaboration.
204k – 255k/yr
Remote9+ YOEData Engineering
Staff Software Engineer
RipplingSeattle, WA +2
Build an end-to-end analytics and business intelligence Data Cloud platform at Rippling, replacing customer data lakes, warehouses, and pipelines with integrated ingestion, transformation, lineage, catalogs, and visualization. Develop large-scale data systems using Python, Trino, Iceberg and Temporal; explore ML/LLMs for automated insights.
189k – 315k/yr
Hybrid8+ YOEData Engineering
Staff Data Engineer
CheckrDenver, CO +1
Staff Data Engineer building and evolving Checkr's centralized people data platform and pipelines that power all AI verification products. Requires 10+ years experience with large-scale data platforms, PySpark, Python, SQL, Kafka, Spark, Iceberg and AWS services; will mentor juniors and own architecture.
166k – 230k/yr
Hybrid10+ YOEData Engineering
Staff+ Software Engineer, Databases
AnthropicSan Francisco, CA +2
Build and scale the core database infrastructure powering Claude at Anthropic, including data plane/control plane, data movement (CDC, migrations), and caching systems that support millions of users and frontier AI research across multi-cloud environments. Requires deep expertise in distributed databases and production storage systems.