Applied AI Data Analyst
The Applied AI Data Analyst evaluates LLM-driven workflows and forecasting models, curates golden datasets, monitors performance, and performs root-cause analysis on complex financial and operational data. The role requires a graduate STEM degree, 2–4 years of quantitative or AI evaluation experience, Python, SQL, and enterprise data infrastructure expertise.
About the job
Responsibilities
- Analyze edge cases, model predictions, hallucination rates, and extraction accuracy for LLM-driven document-processing systems and agentic workflows.
- Perform root-cause analysis on inconsistent financial documents and fragmented enterprise systems to determine why extraction or time-series models succeeded or failed.
- Define, curate, and maintain gold-standard evaluation datasets for automated backtesting of AI agents and demand-forecasting models.
- Build dashboards, data transformations, and evaluation harnesses to monitor system health, model drift, and AI performance metrics across release cycles.
- Partner with applied AI engineers, product managers, and business operations managers to turn real-world problem definitions into quantitative product insights.
Requirements
- Graduate degree (M.S. or Ph.D.) in Computer Science, Data Science, Statistics, Mathematics, Physics, or a related STEM field.
- 2–4 years of experience in a quantitative data analyst or AI evaluation role working closely with AI/ML R&D teams.
- Proficiency in Python and hands-on experience with AI/ML models.
- Strong SQL skills and experience with enterprise data infrastructure, including PostgreSQL, Snowflake, dbt, and AWS.
- Ability to translate noisy, unstructured document data into rigorous evaluation insights for engineers.
Nice to Have
- Experience with LLM observability, tracing, and evaluation frameworks such as LangSmith, Arize AI, or TruLens.
- Proficiency with agentic workflows or multi-agent orchestration.
- Familiarity with demand forecasting, time-series models, or statistical modeling.
- Comfort working in a fast-moving, high-growth startup environment.
Compensation & Benefits
- Equity ownership.
- Fully paid employee health coverage through Aetna.
- Dental and vision coverage through Guardian.
- Carrot Fertility Pro fertility and family-forming support.
- 12 weeks of paid parental leave.
- Unlimited PTO and regular four-day holiday weekends.
- 401(k) through Vestwell.
- Paid relocation support.
- Fully equipped workspace, including a $200 setup stipend.
- Catered Friday lunches, team dinners, unlimited coffee, and snacks.
Skills
Python, SQL, Ai/Ml Models, Postgres, Snowflake, dbt, AWS, Llm Evaluation, Langsmith, Arize Ai, Trulens, Agentic Workflows, Demand Forecasting, Time-Series Models, Statistical Modeling
Similar jobs
Data Analytics jobs
Valon
New York, NY
Own end-to-end data analytics and infrastructure projects for SaaS customers, including migrations, reporting pipelines, dashboards, and business analysis. Requires 2–4+ years of analytics experience, expert SQL, dbt, BI tooling, and strong customer-facing skills.
Ramp
New York, NY
The Credit Risk Associate develops and optimizes credit strategies across limits, payment speed, collections, and decisioning workflows. The role requires credit risk or quantitative strategy experience, SQL or Python proficiency, demonstrated AI fluency, and strong cross-functional judgment.
PrizePicks
United States
Analyzes high-volume transactional data to identify fraud trends, quantify risk, and improve fraud controls while protecting legitimate players. The role requires advanced SQL, dashboarding and visualization skills, fraud or risk analytics experience, and strong communication with cross-functional stakeholders.
Shield AI
Dallas, TX
The Reliability Analyst builds configuration-aware reliability datasets, metrics, dashboards, and reporting for complex aircraft systems. The role analyzes failures and maintenance data, identifies fleet degraders, and supports corrective-action decisions across engineering, quality, and fleet-support teams.
OpenX
United States
Analyzes complex business and financial data, builds automated reporting tools, and delivers actionable insights to stakeholders across the company. Requires 1–3 years of analytical experience, strong SQL and quantitative skills, and a bachelor's degree in a quantitative field.