Staff Software Engineer, Data Warehouse
Staff Software Engineer responsible for architecting, building, and operating Commure’s data warehouse platform, including CDC, lakehouse, query, transformation, and analytics layers. Requires 6+ years of software engineering experience and broad expertise across modern production data infrastructure.
About the job
Responsibilities
- Own the data warehouse platform end-to-end, including CDC pipelines, data lake, query and serving layers, transformation, and analytics tooling.
- Design and operate low-latency, high-fidelity CDC pipelines using Debezium and Kafka, Redpanda, or an equivalent streaming backbone.
- Architect data lakes on object storage using Iceberg, Delta Lake, or Hudi with Parquet for batch and streaming workloads.
- Run and scale StarRocks or comparable MPP/lakehouse engines, including schema design, materialized views, ingestion, tuning, and cost/performance optimization.
- Build dbt transformation models, tests, documentation, and a semantic layer for consistent metrics.
- Establish orchestration, CI/CD, observability, and data-quality tooling.
- Partner with Security and Compliance on PHI/PII handling, access controls, lineage, and auditability for HIPAA and SOC 2 compliance.
- Define schema contracts, ingestion patterns, and self-service tooling for product and analytics teams.
Requirements
- 6+ years of software engineering experience, including substantial experience building or operating data platforms at scale.
- Experience with CDC, streaming, data lake formats, MPP or lakehouse query engines, and dbt.
- Fluency in SQL, schema design, query optimization, and cost and latency trade-offs for large datasets.
- Experience operating production data infrastructure, including orchestration, observability, on-call, data quality, and incident response.
Nice-to-haves
- Production experience with Debezium, StarRocks, and dbt.
- Experience with semantic layers such as dbt Semantic Layer or Cube, and data catalogs or lineage tools such as DataHub, OpenMetadata, or Amundsen.
- Experience with HIPAA-regulated data, including PHI handling, de-identification, and access governance.
- Experience supporting AI/ML workloads, including feature stores, training-set curation, embedding pipelines, or retrieval systems.
- Experience across AWS, Google Cloud, and Azure; infrastructure as code with Terraform or Pulumi; and Kubernetes controllers.
Skills
Debezium, Kafka, Redpanda, Iceberg, Delta Lake, Hudi, Parquet, Starrocks, dbt, SQL, Airflow, Dagster, Terraform, Kubernetes, HIPAA
Similar jobs
Data Engineering jobs
Axion
San Francisco, CASenior individual contributor responsible for architecting and scaling production data ingestion systems that integrate complex enterprise sources into reliable datasets. Requires 5+ years of backend engineering experience, strong Python, cloud, Kubernetes, Postgres, and data integration expertise.
Airbnb
United States
Staff Software Engineer leading development of data catalog and metadata infrastructure for discovery, governance, lineage, and quality across Airbnb’s data ecosystem. Requires 9+ years of software engineering experience focused on data infrastructure and strong programming and distributed data technology skills.
Coinbase
United States
This staff-level data engineer will architect and operate low-latency market data infrastructure, including feed handling, normalization, distribution, and exchange connectivity. The role requires at least five years of backend engineering experience and strong Java or C++ expertise with high-throughput messaging and market data protocols.
Airbnb
United States
Staff Software Engineer leading design and development of large-scale batch and real-time data pipelines and ML infrastructure to power GenAI/LLM products and features for Airbnb's Messaging, Notifications, and Connectivity organization. Requires 9+ years experience building production ML systems and cross-functional collaboration.
OpenAI
San Francisco, CAAnalytics Engineer supporting Go-to-Market teams by building scalable data models, metrics, pipelines, visualizations, and self-service products. The role requires 10+ years of data experience, deep SQL expertise, Python proficiency, and strong business judgment.