AI-Native Data Platform Engineer
Designs and owns canonical data foundations, ingestion pipelines, and AI-ready schemas for financial AI systems in wealth management. Requires 5+ years data platform experience, SQL/Python expertise, custodial data knowledge, and AWS proficiency.
About the job
Responsibilities
- Design scalable ingestion pipelines across custodians (Schwab, Fidelity, Pershing, etc.) and internal financial systems
- Build and evolve canonical models for accounts, positions, transactions, balances, corporate actions, and household hierarchies
- Define financial data ontology and enforce strong data contracts across services
- Implement reconciliation frameworks and golden-source resolution across multi-vendor datasets
- Engineer AI-ready data layers optimized for embeddings, vector search, and RAG architectures
- Structure financial datasets to improve prompt reliability and LLM output consistency
- Architect closed-loop, agent-driven systems that monitor, reason over, and autonomously remediate data inconsistencies
- Implement observability, lineage, governance, and fine-grained access controls across regulated datasets
Requirements
- 5+ years building production-grade data platforms
- Deep SQL expertise and strong Python for data engineering
- Experience designing canonical schemas and resolving vendor data inconsistencies
- Strong understanding of custodial financial data (positions, trades, balances, performance, corporate actions)
- Familiarity with embeddings, vector databases, and retrieval architectures
- Exposure to prompt engineering and structured context design for LLM systems
- Knowledge of MLOps fundamentals (versioning, monitoring, reproducibility)
- Comfortable with AWS data services (S3, Lambda, ECS, Glue, Redshift, OpenSearch) and event-driven orchestration
- Strong ownership mindset and systems-level thinking
Nice-to-Haves
- Wealth management or capital markets background
- Experience integrating OpenAI or Anthropic APIs into production systems
- Experience designing retrieval schemas for AI agents
- Experience with authorization and policy platforms (e.g., OSO, Auth0)
- Experience implementing fine-grained access control for AI-driven systems
- Familiarity with GitHub-based CI/CD workflows and automation
- Experience with data governance, lineage, and compliance controls
Benefits
- Full health benefits + 401(k) matching & Roth IRA options
- Unlimited PTO
Skills
SQL, Python, AWS, Redshift, Glue, S3, AWS Lambda, Opensearch, Embeddings, Vector Databases, RAG, MLOps, Prompt Engineering
Similar jobs
Data Engineering jobs
Bevi
Boston, MA
Build and maintain dbt models, Snowflake semantic layers, and ingestion pipelines across business functions while improving data quality and resilience. The role requires 4–6 years of analytics or data engineering experience, strong dbt and SQL expertise, and a quantitative bachelor's degree.
Anthropic
San Francisco, CABuild and govern quote-to-cash data models and products integrating Salesforce, CPQ, billing, and finance systems. The role requires 5+ years of data engineering experience, strong SQL and Python skills, and expertise in self-service analytics for GTM teams.
Cloudflare
Atlanta, GABuild and scale distributed data platforms, database systems, delivery services, and APIs, with emphasis on reliability, performance, observability, and data integrity. Requires 3+ years of software development experience with distributed systems and databases; Golang experience is preferred.
Astera
Emeryville, CA
Oversee the lifecycle, quality, governance, and publication of research data across scientific programs. The role requires 3–5+ years of research data-management experience, strong metadata and FAIR-data expertise, and the ability to collaborate with researchers and engineers.
Upside
Washington, DCOwn and evolve trusted data models for Marketing and Product use cases, from design and testing through monitoring and documentation. The role requires 3–5 years of data or analytics engineering experience, strong SQL and Python, dbt expertise, and Snowflake or comparable warehouse experience.