Software Engineer II, Big Data, tvScientific
Build and scale AWS-based data infrastructure, pipelines, knowledge graphs, and APIs for a CTV performance advertising platform. The role requires production data engineering experience with Spark, Scala, AWS, SQL, and large-scale services, plus a bachelor's degree.
About the job
Responsibilities
- Design and implement robust data infrastructure in AWS using Spark with Scala.
- Evolve core data pipelines to scale efficiently.
- Store data in optimal engines and formats based on performance and cost requirements.
- Collaborate with cross-functional teams to design data solutions that meet business needs.
- Design and implement knowledge graphs exposed through batch processing and APIs.
- Leverage and optimize AWS resources for scale.
- Collaborate with Data Science and Product teams.
- Deliver scalable, efficient data infrastructure and reliable data assets and APIs.
- Implement automated data quality checks.
Requirements
- Production data engineering experience.
- Proficiency in Spark and Scala, preferably with experience building Spark data infrastructure using Scala.
- Experience delivering significant technical initiatives and reliable, large-scale services.
- Experience delivering APIs backed by relationship-heavy datasets.
- Familiarity with data lakes, cloud warehouses, and storage formats.
- Strong proficiency in AWS services.
- Expertise in SQL for data manipulation and extraction.
- Excellent written and verbal communication skills.
- Bachelor's degree in Computer Science or a related field.
- Ability to use AI to improve speed and quality in day-to-day work.
- Ability to critically evaluate and verify AI-assisted work through testing, source-checking, data validation, or peer review.
- High integrity and ownership when handling sensitive data and final deliverables.
Nice-to-haves
- Adtech experience.
- Data governance experience, including data quality, metadata management, and access controls.
- Understanding of privacy-by-design principles and sensitive or regulated data.
- Familiarity with Apache Iceberg and Delta.
- Experience building a Data Engineering function.
- Experience working with Data Science teams on machine learning pipelines.
Compensation
- Base salary range: $123,696–$254,667 USD.
- Equity eligible.
- Benefits are available for the position.
Skills
AWS, Spark, Scala, SQL, Data Pipelines, Data Lakes, Cloud Warehouses, Knowledge Graphs, APIs, Apache Iceberg, Delta Lake, Data Governance, Machine Learning Pipelines, Data Quality
Similar jobs
Data Engineering jobs
Bevi
Boston, MA
Build and maintain dbt models, Snowflake semantic layers, and ingestion pipelines across business functions while improving data quality and resilience. The role requires 4–6 years of analytics or data engineering experience, strong dbt and SQL expertise, and a quantitative bachelor's degree.
Stuut
San Francisco, CABuild and own Stuut’s foundational data platform, including ingestion pipelines, canonical models, semantic layers, and observability. The role requires 3+ years of production data pipeline experience with Python, SQL, cloud warehouses, and ETL/ELT tooling.
Ontic
United States
Build scalable analytics engineering infrastructure, SaaS data models, and AI-enabled workflows that support enterprise decision-making. The role requires 3–6 years of hands-on analytics or data engineering experience, strong SQL and modern data modeling expertise, and cloud data warehouse experience.
Underdog Fantasy
United States
Build and operate the data platform supporting automated regulatory reporting for a prediction markets business. The role combines SQL and dbt development, end-to-end data investigation, automated validation, and cross-functional ownership under strict deadlines.
Attentive
United States
Build and operate distributed data applications powering large-scale audience segmentation and real-time personalization. The role requires 2–4 years of software engineering experience, backend development skills, and familiarity with databases, algorithms, and distributed systems.