Senior Data Engineer

Role Responsibilities

• Design, build, and operate scalable data pipelines using batch and real-time processing technologies such as Apache Spark, Kafka, Flink, or managed cloud streaming services to process terabytes of data daily.
• Build data infrastructure that ingests real-time events and stores them efficiently across databases, data warehouses, and data lakes within AWS.
• Establish and enforce data contracts with backend engineering teams by implementing schema management, data quality checks, and monitoring to ensure pipeline reliability.
• Make data accessible and consumable for operational services, analytics platforms, and data-intensive product features, balancing latency, freshness, and accuracy requirements.
• Collaborate closely with backend engineers, machine learning engineers, and product partners to understand data access patterns, system constraints, and quality expectations.
• Take ownership of significant portions of the data platform architecture, driving design decisions and technical prioritization.
• Develop tools, frameworks, and recommended patterns that enable rapid development of data products and consistent pipeline deployments.
• Mentor engineers on data engineering best practices and raise the overall quality bar across the organization.
• Stay current with emerging technologies in data processing and infrastructure, evaluating their applicability and impact on Fetch systems.

Recommendation Systems Team Focus

• Building and maintaining feature store infrastructure to support efficient feature development, discovery, and reuse across recommendation models.
• Designing and operating low-latency feature serving systems that power real-time recommendation APIs for both training and inference workloads.
• Implementing monitoring and quality checks to ensure feature freshness, accuracy, and consistency.
• Collaborating with ML engineers to understand feature access patterns, model requirements, and latency and throughput needs.

Minimum Requirements

• 6+ years of professional experience in data engineering, building and operating production data systems at scale.
• Proven experience designing, building, and maintaining scalable batch and real-time data pipelines capable of processing terabytes of data daily.
• Hands-on experience with modern data processing frameworks such as Apache Spark, Kafka, Flink, Open Table Formats, and modern OLAP databases.
• Strong foundation in data architecture principles, including data modeling, schema design, and tradeoffs between latency, reliability, and cost.
• Proficiency in at least one modern programming language such as Go, Python, Java, or Rust, along with strong SQL skills.
• Experience with Infrastructure as Code tools such as Terraform or CloudFormation in a production environment.
• Familiarity with CI/CD processes and modern software development lifecycle practices, with an emphasis on shipping incrementally and improving systems over time.
• Experience implementing data quality controls, including validation, monitoring, and anomaly detection.
• Ability to take ownership of projects with guidance, driving designs from initial architecture through implementation and adoption.
• Comfort presenting technical designs, participating in peer reviews, and constructively challenging decisions.
• Strong collaboration skills with experience working closely with software engineers, machine learning engineers, data analysts, and product partners.
• Undergraduate or graduate degree in Computer Science, Data Science, Engineering, Mathematics, Statistics, or a related quantitative field, or equivalent practical experience.

Compensation

Base salary range: $149,523 - $206,578. Includes equity and benefits (401k match up to 4%, comprehensive health benefits for humans and pets).