Senior Software Engineer - Data Platform
Build and operate scalable data platform services, pipelines, SDKs, and self-service tooling that make internal and third-party data accessible for analytics, machine learning, and product experiences. Requires 5+ years of backend engineering experience and production expertise with data infrastructure.
About the job
Responsibilities
- Design, develop, and operate foundational data services spanning cloud data warehouses, data lakes, Airflow orchestration, batch processing with Spark and SQL, Kafka streaming, query federation, and real-time event aggregation stores.
- Build and maintain data integration and processing SDKs used by internal services and product teams.
- Deliver self-service applications for managing, monitoring, and troubleshooting data pipelines.
- Architect data security and observability services, including access controls, data quality tracking, dataset cataloging, lineage, and usage auditing.
- Partner with data analysts, ML engineers, and security and compliance teams to create reusable, scalable data patterns and microservices.
Requirements
- 5+ years of backend software engineering experience, including production data systems or pipelines.
- Proficiency in Python, Go, or Java for backend development and scalable data services.
- Experience with scale-out architectures, caching layers, key/value stores, and columnar storage systems.
- Production experience with SQL, Airflow, Spark, Kafka, or similar processing and orchestration frameworks.
- Experience with data security, access control, or observability systems, including cataloging, lineage, or quality monitoring.
- Responsible use of generative AI with human oversight.
Compensation
- Annual base salary: $186,065–$218,900 USD.
- Additional compensation may include equity, bonus eligibility, and benefits.
Skills
Python, Go, Java, SQL, Apache Airflow, Spark, Apache Kafka, Data Warehousing, Data Lakes, Data Pipelines, Data Security, Data Observability, Cloud Storage, Key-Value Stores, Columnar Storage
Similar jobs
Data Engineering jobs
Square
United States
Build and operate trusted, scalable data-platform capabilities spanning warehousing, orchestration, governance, BI, and developer tooling. The role requires senior software engineering experience, strong Python skills, modern data-platform expertise, and Terraform proficiency.
Otter
Mountain View, CA
Build and own foundational data models, reliable pipelines, and data platforms that support analytics and business decision-making. The role requires 5+ years of data engineering experience, strong Python and SQL skills, cloud data warehouse expertise, and a bachelor's degree or equivalent experience.
Jump
United States
Build and operate Jump’s Snowflake and dbt data platform, including ELT pipelines, governance, testing, monitoring, and analytics models. The role requires 6+ years of data engineering experience, production-grade SQL and Snowflake expertise, deep dbt knowledge, and strong Python and software engineering practices.
Plaid
New York, NY
Build and own large-scale data models, batch and real-time pipelines, and data infrastructure that provide reliable datasets and insights across Plaid. The role requires 4+ years of data engineering experience, strong SQL and Python skills, and expertise with modern warehouses, lakes, and orchestration tools.
Plaid
Seattle, WA
Builds and owns scalable SQL/Python data pipelines, golden datasets, and workflows using DBT, Airflow, Redshift for large-scale data (500TB+). Collaborates cross-functionally to enable data-driven decisions at Plaid. Requires 4+ years data engineering experience.