Senior Data and AI Platform Engineer

Responsibilities

• Design, build, and maintain data platforms and pipelines that support analytics, data science, BI and AI across Jellyfish.
• Lead the evolution of Jellyfish’s Databricks Lakehouse platform, helping define the architecture, governance model, development patterns, and operating practices that make the platform reliable, scalable, and easy to use.
• Enable enterprise-wide agentic analytics within Databricks with trusted datasets, semantic definitions, well-managed context, and well-governed agentic access.
• Mature core Databricks platform capabilities, including Unity Catalog, governed data access, data lineage, metadata management, compute patterns, environment management, and platform observability.
• Build and maintain ingestion pipelines that bring high-value data into the Lakehouse.
• Partner on data flows that send trusted data to other internal systems.
• Collaborate with data scientists, analysts, product managers, engineering leaders, customer success leaders, go-to-market leaders, and other stakeholders to understand analytical needs and design durable platform solutions.
• Create standards and reusable patterns for data modeling, documentation, observability, testing, governance, and AI-readiness across the data platform.
• Develop tools and processes to monitor data platform health, pipeline reliability, cost, performance, usage, and trust.
• Provide technical leadership as a senior individual contributor by setting architectural direction, raising engineering standards, mentoring teammates, and helping the team make high-quality technical decisions.
• Stay current with emerging Databricks and AI capabilities, evaluate where they can create real value for Jellyfish, and help turn promising ideas into production-ready platform capabilities.

Requirements

• Deep experience in data engineering, data platform engineering, analytics engineering, or related roles.
• Designed, built, and operated reliable data platforms or large-scale data pipelines in production.
• Strong experience with Databricks or similar lakehouse/data platform technologies.
• Understand how to build governed, well-modeled data assets that can support BI, analytics, data science, and AI use cases.
• Experience with data ingestion, transformation, orchestration, testing, monitoring, and data quality practices.
• Advanced SQL skills and experience working with multiple database and warehouse technologies.
• Strong programmer with experience building production-grade systems in Python or similar languages.
• Understand the importance of metadata, documentation, lineage, access control, and semantic context in making data trustworthy and usable.
• Excited about AI and agentic analytics, but understand that successful AI depends on strong data foundations, clear definitions, governance, evaluation, and operational discipline.
• Comfortable working with technical and non-technical stakeholders, translating ambiguous needs into durable platform capabilities.
• Operate as a senior individual contributor: lead through architecture, judgment, communication, influence, and execution without needing to be a people manager.
• Strong communication skills and enjoy working as part of a cross-functional team.

Nice-to-Haves

• Experience with Databricks Unity Catalog, Databricks SQL, Lakehouse architecture, Delta Lake, Databricks Workflows, Databricks Apps, Genie, or related Databricks AI/BI capabilities.
• Building platforms for self-service analytics, governed BI, semantic layers, metrics layers, or AI-assisted analytics.
• Designing data platforms that support LLMs, agents, retrieval-augmented generation, MCP, or other AI-enabled workflows.
• Infrastructure-as-code and platform automation tools such as Terraform, Databricks Asset Bundles, CI/CD pipelines, or similar technologies.
• dbt and modern analytics engineering practices.