Senior Data Engineer
Senior Data Engineer building and owning large-scale ETL pipelines at Tatari to power reporting and measurement products from high-volume TV advertising data. Requires 5+ years experience with Python, SQL, Spark, Airflow, and strong data quality practices in a cross-functional environment.
About the job
Responsibilities
- Design, build, and own large-scale data pipelines that transform high-volume raw data into curated, production-grade datasets for reporting and measurement products.
- Take data products from requirements through tech spec, implementation, validation, and production rollout.
- Partner closely with data science on methodology, backend and frontend engineers on serving layers, and product on requirements.
Requirements
- 5+ years of dedicated experience designing, building, and operating production ETL pipelines at scale.
- Strong programming skills in Python (or similar) and SQL.
- Deep hands-on experience with Spark/PySpark, Databricks, and Delta Lake.
- Experience with workflow orchestration tools such as Airflow, including DAG design, backfills, and failure recovery.
- Strong data modeling skills for analytics, including curated/gold-layer datasets, schema design, partitioning, and data contracts for downstream services.
- Rigorous approach to data quality: validation, reconciliation, and methodology comparison.
- Experience debugging distributed data systems, including performance tuning, memory issues, and optimizations.
- Strong written communication and cross-functional collaboration skills.
Nice-to-Haves
- Ongoing interest in agentic developer workflows: testing agent harnesses, building them, agent loop engineering, and similar emerging paradigms.
- Experience with OLAP stores (e.g., ClickHouse).
Compensation and Benefits
- Total compensation: $170,000 - $200,000 annually.
- Equity compensation.
- Health insurance coverage for you and your dependents.
- 401K, FSA, and commuter benefits.
- $150 monthly spending account.
- $1,000 annual continued education benefit.
- $500 Newbie Productivity Perk.
- Unlimited PTO and sick days.
- Monthly Company Wellness Day Off.
- Snacks, drinks, and catered lunches at the office.
- Team building events.
- Hybrid RTO of 2 days per week in office.
Skills
Python, SQL, Spark, Pyspark, Databricks, Delta Lake, Airflow, ETL, Data Modeling, Data Quality, Olap, ClickHouse
Similar jobs
Data Engineering jobs
Crusoe
San Francisco, CA
Build foundational data platform infrastructure for analytics, machine learning, and cloud operations. The role requires 5+ years of software or data infrastructure engineering experience, strong Python and systems programming skills, distributed systems knowledge, and advanced SQL expertise.
Vercel
San Francisco, CAOwns the Finance data infrastructure supporting billing, usage-based revenue, forecasting, reporting, and close. The role requires production data engineering experience, strong SQL and Python, dbt and orchestration expertise, Finance-domain fluency, and the ability to mentor engineers and partner with business stakeholders.
Bevi
Boston, MA
Own and evolve Bevi’s end-to-end data platform, from ingestion and IoT modeling through governed self-service analytics and AI access. The senior individual contributor will architect scalable streaming and batch systems, establish governance and observability, and provide technical leadership across the Data & Data Science organization.
Clay
New York, NYBuild customer-facing data products and shared platform systems that transform conflicting, constantly changing sources into reliable, searchable information. The role requires 8+ years of hands-on engineering experience, strong Python and SQL skills, and ownership of product quality, reliability, and delivery.
Rippling
San Francisco, CA
Senior Software Engineer building and operating broad data platform infrastructure for data, analytics, ML, AI, and agent workflows. Requires 5+ years of software engineering experience with distributed systems, production data platforms, cloud infrastructure, and systems design.