Senior Data Engineer
Senior Data Engineer owning architecture and technical vision for ClickUp's data platform. Build scalable pipelines on AWS serverless, Snowflake, and dbt; design AI/ML infrastructure; drive cost optimization, standards, and mentor engineers. Requires deep expertise in cloud data systems and large-scale distributed architectures.
About the job
Responsibilities
- Own the technical architecture of ClickUp's data platform, making design decisions that balance scalability, cost, reliability, and velocity.
- Define and drive the technical roadmap for data infrastructure in partnership with leadership.
- Design systems at scale: build frameworks, abstractions, and patterns that other engineers use daily.
- Lead complex, cross-team technical initiatives spanning data engineering, analytics engineering, data science, and data analytics.
- Drive cost optimization across cloud infrastructure and compute.
- Build and evolve our data pipelines using AWS serverless (Lambda, Fargate, Step Functions, Kinesis, S3, DynamoDB, Aurora), Snowflake, and dbt.
- Establish and champion engineering standards: observability, testing, CI/CD, code review, and documentation practices.
- Design and maintain infrastructure for AI/ML workloads, including LLM frameworks, feature pipelines, training data systems, and model monitoring.
- Mentor senior engineers, provide technical guidance through design reviews, and raise the overall engineering quality of the team.
- Influence org-wide technical decisions and represent data engineering in company-level architecture discussions.
Requirements
- Significant professional experience in data engineering or backend/infrastructure engineering, with at least 3 years operating at a senior or staff level.
- Proven track record of owning architecture for data platforms or large-scale distributed systems.
- Deep expertise in AWS cloud services (Lambda, Fargate, Step Functions, S3, Kinesis, DynamoDB, Aurora) and infrastructure as code (Terraform and/or CDK).
- Expert-level SQL and Snowflake (or equivalent cloud data warehouse) knowledge, including performance tuning and cost optimization.
- Strong experience with dbt and modern ELT/ETL patterns at scale.
- Advanced Python skills with emphasis on building reusable libraries, frameworks, and tooling.
- Hands-on experience with orchestration frameworks (Airflow, Dagster, or Prefect) in production environments.
- Experience building data infrastructure for AI/ML: feature stores, training pipelines, embedding pipelines, model serving, or LLM integration.
- Deep understanding of streaming and event-driven architectures (Kinesis, Kafka, or equivalent).
- Mastery of CI/CD, Git workflows, containerization (Docker), and deployment automation.
- Strong communication skills: ability to write technical RFCs, influence without authority, and translate complex trade-offs for non-technical stakeholders.
- Track record of mentoring and growing engineers, with a multiplier mindset.
Nice-to-Haves
- Experience operating data platforms at high scale (petabyte-level warehouses, millions of events/sec).
- Familiarity with data mesh or data product paradigms.
- Experience with FinOps practices and cloud cost management at scale.
- Prior experience in a technical leadership role without direct reports (staff/principal IC track).
- Contributions to open-source data tools or technical communities.
Skills
AWS, Snowflake, dbt, Terraform, Python, SQL, Airflow, Dagster, Prefect, Docker, Kafka, Kinesis, AWS Lambda, S3, DynamoDB
Similar jobs
Data Engineering jobs
Mark43
Boston, MA
Own and scale Mark43’s production database infrastructure, focusing on MySQL administration, monitoring, performance, reliability, upgrades, and recovery. The role requires at least seven years of production database experience, cloud infrastructure expertise, and familiarity with Terraform.
ZoomInfo
Waltham, MA
Build and operate large-scale data acquisition pipelines, distributed processing systems, and production Java services across batch and streaming workloads. The role requires 5+ years of backend or data engineering experience, strong distributed-systems expertise, and proficiency with cloud data technologies.
ZoomInfo
United States
Builds and operates large-scale web crawling, extraction, and data engineering infrastructure processing billions of pages. The role requires 5+ years of software engineering experience, strong distributed-systems fundamentals, and proficiency with Java or Python, cloud platforms, Kubernetes, and ETL technologies.
Lyft
Toronto, Canada
Leads architecture for offline experimentation and route simulation while building reliable, scalable data pipelines and backend services. The role requires 5+ years of backend or data engineering experience, distributed-systems expertise, strong SQL and Spark skills, and proficiency with modern cloud infrastructure.
Square
California
Senior individual contributor who architects and builds end-to-end people data systems, predictive models, and AI-agent workflows. Requires 8+ years of experience across data engineering and data science, with expertise in Python, SQL, machine learning, sensitive HR data, and agentic AI.