Data and AI Lead, Finance
Leads architecture and delivery of complex finance data pipelines, AI use cases, and internal applications while partnering with Accounting, FP&A, and Finance leadership. Requires 12+ years of data engineering or finance systems experience, strong SQL/Python and Databricks expertise, and deep finance-domain knowledge.
About the job
Responsibilities
- Design and develop ETL pipelines using Databricks SQL and Python/PySpark to improve reporting, automate journal entries, and transform finance and accounting processes across revenue, expenses, equity, commissions, and tax.
- Architect, build, and own complex finance data pipelines in the finance data lake using Databricks Jobs and Lakeflow Declarative Pipelines, including data validation and reconciliations.
- Lead the technical design of AI use cases for Finance and Accounting, including forecasting automation, anomaly detection, and natural-language interfaces to financial data.
- Build and maintain internal finance applications, including Databricks Apps, Genie Agents, and AI/BI dashboards.
- Design and deliver curated finance datasets and enforce row-level security, data access policies, and Unity Catalog governance standards.
- Define and champion coding standards, data-modeling conventions, documentation practices, and testing frameworks.
- Manage Git-based version control, pull-request reviews, and CI/CD pipelines using Declarative Automation Bundles and GitHub Actions to satisfy SOX change-management requirements.
- Lead technical scoping and solutioning for Accounting, FP&A, Internal Audit, and Procurement requirements.
- Serve as the primary technical contact during financial close, ensuring data accuracy and timely resolution of pipeline issues.
- Partner with IT and Engineering on system integrations, providing technical requirements and leading user acceptance testing.
- Mentor junior and mid-level engineers through code reviews, pairing, and design discussions.
- Identify architectural debt, performance bottlenecks, and tooling gaps, and drive resolutions.
Requirements
- 12+ years of experience in data engineering, analytics engineering, or finance systems, with ownership of complex production-grade pipelines.
- Deep proficiency in SQL and Python, with hands-on experience using Apache Spark and Databricks.
- Experience building and maintaining ELT/ETL pipelines from financial source systems such as NetSuite, Salesforce, Stripe, or Zuora into a centralized data lake.
- Strong understanding of finance and accounting concepts, including close processes, revenue recognition, intercompany accounting, chart of accounts, and financial reporting.
- Ability to translate ambiguous Finance requirements into well-architected, maintainable technical solutions.
- Ability to lead technical discussions with engineering peers and non-technical Finance stakeholders.
- Experience building Finance-facing dashboards, self-service BI products, and executive reporting layers.
- Familiarity with AI/ML concepts and applying them to Finance workflows.
Nice to Have
- Experience at a high-growth SaaS or cloud infrastructure company.
- Hands-on experience with AI/BI tools, Genie, or LLM-powered applications.
- Experience with Declarative Automation Bundles or CI/CD for finance data lake pipelines.
- CPA, CFA, or formal finance/accounting background.
Skills
SQL, Python, Pyspark, Spark, Databricks, Databricks Jobs, Lakeflow Declarative Pipelines, ETL, Data Lake, Unity Catalog, GitHub Actions, CI/CD, AI/ML, Ai/Bi Dashboards, Llm Applications
Similar jobs
Data Engineering jobsBuild and operate scalable lakehouse infrastructure, streaming and CDC pipelines, query systems, and self-serve BI capabilities. Requires 5+ years of data engineering experience, strong Kubernetes and infrastructure-as-code expertise, and hands-on experience with distributed data platforms.
Build and operate distributed systems powering Apache Pinot’s real-time analytics platform at massive scale. The role requires strong distributed-systems expertise, end-to-end delivery ownership, and a focus on reliability, observability, and performance.
Senior data platform engineer who scales infrastructure, automates data delivery, builds AI-enabled analytical tools, and leads cross-functional engineering initiatives. Requires 4+ years of data infrastructure experience, strong Kafka and distributed-systems expertise, and proficiency in Python, Scala, cloud platforms, and Terraform.
Senior Software Engineer building reliable connectors and high-volume data pipelines that move customer data into warehouses. The role requires strong Java, cloud, database, distributed-systems, and technical leadership experience.
Build and operate scalable enterprise data pipelines, models, and platform infrastructure across the full data lifecycle. The role requires 5+ years of experience, strong SQL and Python, and deep expertise in Snowflake, dbt, and Airflow.