# Data Annotation Specialist, Generalist

**Company:** [Cohere](https://hotfix.jobs/companies/cohere)
**Location:** Remote
**Role:** Data Analytics
**Salary:** CA$62k – CA$62k/yr
**Experience:** 1+ years
**Skills:** Ai Data Annotation, Llm Evaluation, Content Moderation, Quality Assurance, Preference Ranking, Data Labeling, JSON, Csv/Tsv, Markdown, Xml, Yaml, LLMs
**Posted:** 2026-07-24

> This part-time contractor evaluates, annotates, and stress-tests AI model outputs across text, image, and structured data. The role requires at least one year of relevant analytical or annotation experience, excellent written English, strong judgment, and meticulous attention to detail.

## Job Description

## Responsibilities
- Evaluate and rank model outputs for accuracy, helpfulness, tone, and safety, providing clear justifications.
- Stress-test models to identify failure modes, unsafe behavior, and capability gaps, and document reproducible cases.
- Create prompts, responses, and exemplars for training and evaluation datasets.
- Build and apply grading criteria, rubrics, and taxonomies.
- Annotate, audit, and correct text, image, and structured data such as JSON, CSV/TSV, and Markdown.
- Participate in calibration exercises and inter-annotator agreement checks.
- Adapt to experimental and evolving task types.
- Report model performance trends and provide evidence-based feedback to cross-functional partners.

## Requirements
- 1+ years of experience in AI data annotation, LLM evaluation, content moderation, research, or a related analytical role, with exposure to quality assurance and/or preference ranking.
- Experience applying detailed guidelines to complex and ambiguous content.
- Strong contextual and sociocultural judgment, with sensitivity to nuance, tone, and register.
- Comfort with ambiguity and willingness to flag unclear edge cases.
- Strong attention to detail, accuracy, and consistency across high-volume tasks.
- Excellent written English and reading comprehension, including the ability to justify evaluations and write clear prompts and exemplars.
- Ability to work independently in a remote, global, asynchronous environment.
- This is a part-time independent contractor position within Canada, requiring a commitment of 16 hours per week.
- BYOD; applicants must provide their own laptop.
- The work may involve exposure to explicit, offensive, or otherwise inappropriate content.

## Nice-to-haves
- Familiarity with large language model behavior, including hallucination, sycophancy, and instruction-following gaps.
- Fluency in another language.
- Experience with annotation platforms and structured formats such as JSON, CSV/TSV, Markdown, XML, and YAML.

## Compensation
- CAD $30/hour contract rate.
- Independent contractors are not eligible for employee health benefits or other employee benefits.
- Work is project-based, and continuous work is not guaranteed; hours and compensation may fluctuate based on project availability.

## Similar jobs

- [Market Research Manager](https://hotfix.jobs/jobs/3da1715f-68a1-4304-9f4b-aeceaf0957c6) - PointClickCare - Mississauga, Canada - CA$86k – CA$96k/yr
- [Data Analyst — Product Analytics](https://hotfix.jobs/jobs/69894d78-1e75-403c-97eb-02d9dc7d4dca) - Virta Health - Remote - $96k – $125k/yr
- [Data Analytics Partner, People Analytics](https://hotfix.jobs/jobs/d84802a4-4c7c-476c-8772-ad8b0269319f) - Dropbox - Remote - $110k – $168k/yr
- [Model Risk Management Specialist](https://hotfix.jobs/jobs/33ebef34-dac3-4aca-8d43-1f8cbe4aab96) - Kraken - Remote
- [Data Analyst](https://hotfix.jobs/jobs/23a3fe0c-6758-4356-857f-5a868cfbf2bf) - Stripe

**Apply:** https://hotfix.jobs/jobs/7a94427b-293d-4540-8c75-0b6df6708b80
**Canonical:** https://hotfix.jobs/jobs/7a94427b-293d-4540-8c75-0b6df6708b80