Director of Engineering, Lakehouse Platform

Responsibilities

Architecture and Technical Strategy

• Own and evolve the Lakehouse Platform and Infrastructure foundations including storage layer, catalog, Spark query engine, Databricks IaC, platform data pipelines, and releases
• Lead the Lakehouse Developer Platform and DX supporting Tetraflows, Semantic services, and Data Resources
• Evolve Lakehouse architecture across open table formats, partition strategies, schema evolution, governance, and API contracts
• Design for durability: version coupling, artifact deployment safety, and protocol compatibility across platform releases
• Collaborate with platform and infrastructure teams on integration contracts, data pipelines, shared execution roadmaps, and operational excellence standards

Engineering Execution

• Own technical prioritization and delivery across a team of 8+ engineers; drive sprint-level execution and quarterly delivery commitments
• Lead incident response and root-cause analysis for production issues; build systemic fixes
• Make and defend build-vs-buy decisions for Lakehouse components
• Establish engineering standards: testing practices, observability instrumentation, and release safety for data infrastructure

People and Team Development

• Coach engineers at all levels with technical mentorship, growth plans, and direct performance feedback
• Hire and develop senior ICs and tech leads; build team depth to reduce knowledge concentration risk
• Model the builder culture: write code, ship internal tools, and set the bar for technical craft

AI-Forward Development

• Champion AI-assisted development practices across the team
• Identify opportunities to apply AI to data quality, schema inference, anomaly detection, and platform observability
• Contribute to broader AI platform strategy from the Lakehouse data infrastructure layer

Requirements

Required

• 10+ years of engineering experience at top-tier technology organizations, with at least 4 years focused on data engineering or distributed systems at production scale
• Expertise in Lakehouse technologies: Apache Spark, Delta Lake or Apache Iceberg, Databricks or equivalent distributed compute platform
• Experience leading Data Engineering teams (6+ engineers) with direct accountability for strategy, execution, operational excellence, and people development
• Strong command of distributed systems, cloud, and modern data stack: columnar and open table formats, query execution engines, partitioning strategies, metadata catalogs, decoupled storage and compute
• Track record of building, shipping, and operating Tier 1 production data platforms
• Cloud-native fluency: AWS, GCP, or Azure; containerization; Infrastructure-as-Code (Terraform or equivalent)
• Clear written and verbal communication for presenting architectural decisions to technical and non-technical stakeholders

Strong Advantage

• Experience in regulated industries (life sciences, healthcare, financial services) with data governance and auditability requirements
• Familiarity with scientific data pipelines, analysis, or instrument data pipelines
• Real-time or near-real-time streaming experience (Kafka, Apache Flink, or equivalent)
• Exposure to AI/ML workflows over Lakehouse data: feature stores, ML pipelines, or vector search infrastructure
• Prior career arc as a senior IC (Staff+ / Principal Engineer) before moving into engineering leadership

Compensation and Benefits

• Competitive compensation
• Stock options in VC-backed Series C company
• 100% employer-paid benefits for eligible employees and immediate family members
• 401(k)
• Unlimited PTO
• Flexible working arrangements
• Company-paid Life Insurance, LTD/STD