Senior Data Engineer
Designs and runs massive-scale data pipelines for ingestion, normalization, enrichment, and delivery across 80M+ companies and 800M+ people. Manages data operations, BPO vendors, partnerships, monitoring, and cost optimization using Python, Dagster, and DuckDB.
About the job
Responsibilities
- Own end-to-end data flows: ingestion, normalization, entity resolution, enrichment, and delivery.
- Stand up monitoring for freshness, completeness, and accuracy; drive RCA and prevention.
- Build internal tools that make data discoverable and usable by engineering and product.
- Recruit, onboard, and manage BPO vendors; negotiate and run data partnerships.
Core Requirements
- Core stack: Python, Dagster, DuckDB
- Pipelines at scale: Building resilient ELT/ETL with strong contracts, idempotency, and lineage.
- Data operations: Set quality bars, manage BPO workflows, and run SLAs with external data partners.
- Serving & access: Position data for production use from serving infrastructure, documentation, and SLAs for internal consumers.
- Cost & performance: Tune storage/compute and keep a sharp eye on unit economics.
- Opinionated: Deep level of understanding of the technological landscape, making both high level system and granular code design decisions based on understanding rather than preference - diving deep on unknown patterns in order to build the best product.
Nice to Have
- Experience with big data, columnar storage formats, vector indexes, and privacy/compliance in data products.
Compensation
- Salary: $165K - $250K
- Equity: $100K-200K equivalent (4 year vest)
Skills
Python, Dagster, Duckdb, ELT, ETL, Entity Resolution, Data Pipelines, Data Monitoring, Columnar Storage, Vector Indexes
Similar jobs
Data Engineering jobs
Deepgram
United States
Own the company’s metric governance program by defining canonical metrics, enforcing them in semantic and catalog systems, improving data quality, and validating AI-agent outputs. Requires 5+ years in analytics or analytics engineering, strong SQL, production semantic-layer ownership, and experience with AI evaluation and data governance.
Underdog Fantasy
United States
Build and own the data platform powering regulatory reporting for a federally regulated prediction-markets business. The role combines dbt and SQL engineering, automated validation, root-cause investigations, audit support, and direct collaboration with regulators and compliance.
Granica
Mountain View, CA
Build and optimize foundational lakehouse infrastructure for AI, spanning metadata, transactions, table maintenance, storage layout, and query performance at massive scale. The role requires senior systems engineering experience with modern lakehouse technologies, columnar formats, cloud object storage, and systems-oriented programming languages.
Vercel
San Francisco, CAOwns the Finance data infrastructure supporting billing, usage-based revenue, forecasting, reporting, and close. The role requires production data engineering experience, strong SQL and Python, dbt and orchestration expertise, Finance-domain fluency, and the ability to mentor engineers and partner with business stakeholders.
Vercel
San Francisco, CAOwns end-to-end GTM data pipelines, transformations, and models that power reliable pipeline, revenue, attribution, and funnel reporting. The role requires senior-level data engineering experience, strong SQL and Python, dbt and orchestration expertise, GTM metric fluency, and stakeholder partnership skills.