Sr. Staff Platform/Data Reliability Engineer, Databricks

Responsibilities

• Own operational excellence for the Databricks platform, including monitoring, alerting, observability, incident response support, and production runbook patterns for data jobs and platform services.
• Define and maintain CI/CD and promotion standards for Databricks assets, including workflows, jobs, notebooks, code packages, infrastructure configuration, and environment promotion from development to production.
• Design and maintain platform standards for job orchestration, cluster and compute policies, service principal usage, environment isolation, and production execution reliability.
• Establish reusable operational templates and enablement patterns for new domains onboarding to Databricks, including logging conventions, job tagging, metadata capture, and support handoff expectations.
• Partner with the Senior Data Engineer to ensure ingestion and medallion patterns are observable, recoverable, cost-aware, and secure in production.
• Work with the cloud and infrastructure team to align Databricks configuration and usage patterns with broader enterprise cloud standards, including commercial and future government-hosted environments.
• Help enforce technical controls for data segregation, access boundaries, and operational compliance in a highly regulated environment.
• Track and improve platform health metrics such as job success rates, incident trends, data pipeline reliability, cost efficiency, and environment drift.
• Document platform standards, operational expectations, and support models so the Databricks platform can scale beyond a small founding team.
• Mentor internal engineers developing platform responsibilities and Databricks operational expertise.

Requirements

• 12+ years of relevant experience in data platform engineering, platform operations, site reliability engineering, or modern cloud data infrastructure.
• Hands-on experience with Databricks or a closely related cloud data platform in production environments.
• Experience designing or operating CI/CD, environment promotion, version control, and deployment automation for data platforms and pipelines.
• Strong understanding of observability, monitoring, alerting, incident management, and reliability engineering.
• Experience with compute policy design, workload isolation, service principals, and secure production execution patterns on cloud data platforms.
• Ability to work effectively in regulated or security-sensitive environments with strong expectations around access control, auditability, and operational discipline.
• Strong collaboration skills and comfort partnering with cloud/infrastructure, security, data engineering, and analytics stakeholders.

Nice-to-Haves

• Databricks certification and/or demonstrated expertise with Delta Lake, Unity Catalog, Workflows, and Databricks Asset Bundles.
• Experience with infrastructure as code and platform automation in enterprise environments.
• Experience supporting commercial and government or otherwise segregated environments with different compliance and access requirements.
• Experience in defense, aerospace, federal, or another regulated industry.