Senior Software Engineer, Data Products
Senior engineer building performant user-facing data products from internal datasets using Python, Databricks, and Postgres while collaborating with platform teams.
Senior Data Engineer
Designs and runs massive-scale data pipelines for ingestion, normalization, enrichment, and delivery across 80M+ companies and 800M+ people. Manages data operations, BPO vendors, partnerships, monitoring, and cost optimization using Python, Dagster, and DuckDB.
165k – 250kNew York, NYData EngineeringOnsite
About the role
Responsibilities
- Own end-to-end data flows: ingestion, normalization, entity resolution, enrichment, and delivery.
- Stand up monitoring for freshness, completeness, and accuracy; drive RCA and prevention.
- Build internal tools that make data discoverable and usable by engineering and product.
- Recruit, onboard, and manage BPO vendors; negotiate and run data partnerships.
Core Requirements
- Core stack: Python, Dagster, DuckDB
- Pipelines at scale: Building resilient ELT/ETL with strong contracts, idempotency, and lineage.
- Data operations: Set quality bars, manage BPO workflows, and run SLAs with external data partners.
- Serving & access: Position data for production use from serving infrastructure, documentation, and SLAs for internal consumers.
- Cost & performance: Tune storage/compute and keep a sharp eye on unit economics.
- Opinionated: Deep level of understanding of the technological landscape, making both high level system and granular code design decisions based on understanding rather than preference - diving deep on unknown patterns in order to build the best product.
Nice to Have
- Experience with big data, columnar storage formats, vector indexes, and privacy/compliance in data products.
Compensation
- Salary: $165K - $250K
- Equity: $100K-200K equivalent (4 year vest)
Skills
PythonDagsterDuckdbELTETLEntity ResolutionData PipelinesData MonitoringColumnar StorageVector Indexes
Similar roles
Data Engineering jobsSenior Software Engineer, Data Governance & Foundations
Instacart is seeking a Senior Software Engineer for the Data Governance & Foundations team. This role involves defining and driving architecture roadmaps for large-scale data ingestion and processing infrastructure, owning end-to-end platform initiatives, and optimizing infrastructure spend.
Senior Analytics Engineer
Owns the data warehouse, semantic layer, and ingestion pipelines using Snowflake, dbt, and Looker. Architects reliable data models, integrates new sources, enables AI workflows, and sets company-wide metrics standards. Requires 5+ years in analytics/data engineering with strong SQL, dbt, and Python.
Senior Software Engineer, Data Engineering
Builds and scales ETL pipelines, designs data schemas, and owns data quality/governance for 10x growth. Requires 5+ years in data pipelines with SQL, Spark, Airflow, Python, and MPP databases like Snowflake/Redshift.