# AI Data Analyst

**Company:** [Glean](https://hotfix.jobs/companies/glean)
**Location:** Bengaluru, India
**Role:** Data Analytics
**Experience:** 3+ years
**Skills:** Data Labeling, Data Analysis, Quality Assurance, Artificial Intelligence, Natural Language Processing, Machine Learning, SQL, Qualitative Analysis, Spreadsheet Workflows, Llm-As-A-Judge
**Posted:** 2026-08-24

> Evaluates AI outputs and maintains benchmark datasets to improve response quality, failure-mode analysis, and release readiness. Requires 3–5 years in data labeling, analysis, QA, or related work, plus experience with AI/ML systems, SQL, and structured qualitative evaluation.

## Job Description

## Responsibilities
- Conduct ongoing human labeling for priority AI workflows, including response quality, task success, MCP and tool use, and end-to-end workflows.
- Triage bad-query reports, downvotes, escalations, and routed quality issues; categorize failure modes and provide analysis to partner teams.
- Perform qualitative analysis of hallucinations, retrieval failures, tool-use issues, weak grounding, and poor workflow completion.
- Follow and improve labeling guidelines and rubrics for consistent, realistic judgments.
- Maintain labeled datasets, golden sets, and regression slices; monitor coverage, drift, leakage, and difficulty distribution.
- Participate in grader calibration and validate LLM-as-a-judge outputs against human labels.
- Partner with Engineering, Product, QA, and evaluation owners on quality reads, benchmark updates, and release-readiness decisions.
- Contribute to recurring quality reporting with failure modes, coverage gaps, quality shifts, and recommendations.

## Requirements
- 3–5 years of experience in data labeling, data analysis, QA, or a related field.
- Strong analytical judgment and attention to detail.
- Experience evaluating AI-generated outputs or working with NLP, search, recommendation, or other ML systems.
- Familiarity with structured qualitative analysis, basic SQL/data retrieval, and spreadsheet workflows.
- Clear written and verbal communication skills.
- Ability to work independently in ambiguous, fast-moving environments and collaborate across functions.

## Compensation & Benefits
- Compensation is determined by location, level, job-related knowledge, skills, and experience.
- Certain roles may be eligible for variable compensation, equity, and benefits.
- Hybrid role requiring four days per week in the office.

## Similar jobs

- [Payments Risk Analyst II](https://hotfix.jobs/jobs/12dae607-a9d6-4cde-aece-8149dd535616) - Coinbase - Hyderabad, India - ₹817k – ₹860k/yr
- [Talent Acquisition Business Analyst](https://hotfix.jobs/jobs/9a29f1bb-485b-4363-a3bf-9a214e01ba08) - Motive - Remote
- [Data Analyst — Product Analytics](https://hotfix.jobs/jobs/69894d78-1e75-403c-97eb-02d9dc7d4dca) - Virta Health - Remote - $96k – $125k/yr
- [Data Analyst – Revenue & Metrics](https://hotfix.jobs/jobs/58d67e41-1829-47d9-a63c-e54df90a5881) - Launchdarkly
- [CVAT Annotation Intern](https://hotfix.jobs/jobs/14110493-4d3a-4245-a884-8e85d3ef189f) - Origin - Remote

**Apply:** https://hotfix.jobs/jobs/16ba3783-834e-4ffe-94d5-a7bb59800095
**Canonical:** https://hotfix.jobs/jobs/16ba3783-834e-4ffe-94d5-a7bb59800095