# Member of Technical Staff, North Modelling

**Company:** [Cohere](https://hotfix.jobs/companies/cohere)
**Location:** London, United Kingdom, Europe, Toronto, Canada
**Role:** ML Engineering
**Skills:** LLMs, Machine Learning, Machine Learning Engineering, AI Agents, Model Evaluation, Evaluation Design, Feedback Loops, Data Curation, Prompting, Model Adaptation, Model Selection, Failure Analysis, Regression Tracking, Privacy-Preserving Logs, Enterprise Ai
**Posted:** 2026-08-19

> Own North’s evaluation strategy and build feedback systems that connect real enterprise workflows, user needs, and production failures to model improvements. The role requires strong applied machine learning judgment and the ability to translate product signals into rigorous, actionable evaluations.

## Job Description

## Responsibilities
- Own the evaluation strategy for North across agent workflows, tool use, enterprise knowledge work, deep research, document creation and editing, and other human-AI interactions.
- Build high-quality evaluations based on user feedback, production failures, privacy-preserving usage logs, internal dogfooding, customer needs, and product goals.
- Create systems that continuously turn product, customer, and user learnings into evaluations.
- Translate evaluation results, customer-derived data, and product context into recommendations for modelling teams.
- Identify model failures, capability gaps, and North-specific needs to guide model selection, patches, and regular model updates.

## Requirements
- Experience improving LLM-powered, agent-powered, or AI-product systems through evaluations, feedback loops, data curation, prompting, model adaptation, or model selection.
- Strong applied machine learning engineering judgment.
- Ability to reason about model behavior, evaluation validity, product outcomes, and production tradeoffs.
- Experience working with users and product teams and translating qualitative signals into actionable measurement.
- Self-directed and practical approach to open-ended technical and product problems.
- Interest in making AI systems useful in real enterprise settings.

## Nice-to-haves
- Experience with representative tasks, precise rubrics, clean data, failure analysis, regression tracking, and identifying misleading metrics.
- Experience evaluating agentic workflows and enterprise knowledge work.

## Compensation & Benefits
- Weekly lunch stipend of $75/£75 or equivalent in local currency.
- Full health and dental benefits, including a mental health budget.
- RRSP matching, 401K, or pension scheme.
- Parental leave top-up for up to six months for either parent.
- Annual enrichment benefits for arts and culture, fitness and wellness, quality time, and workspace improvements.
- Education and learning stipend for conferences, courses, and coaching.
- Six weeks of paid vacation.
- Travel budget for other offices and an annual company offsite.
- Coworking benefit and a $500 home office stipend.

## Similar jobs

- [AI Software Engineer](https://hotfix.jobs/jobs/c9a0e889-a36e-488f-87d0-e9c27ab63bf0) - Rollstack - Remote
- [Applied AI Engineer, Digital Natives](https://hotfix.jobs/jobs/74eb3e31-bab1-479e-90e1-2e172658343a) - OpenAI - London, United Kingdom
- [Software Engineer - AI Automation](https://hotfix.jobs/jobs/5184d049-8c0b-4184-8b13-c810a07b5886) - Coalition Security - Remote - $115k – $168k/yr
- [Agent Engineer](https://hotfix.jobs/jobs/627cc378-6ac6-467e-bbf0-2c07738ae0a5) - Elliptic - London, United Kingdom
- [AI Engineer - New Verticals](https://hotfix.jobs/jobs/4159c72e-536e-4211-969c-6bcb9ad805fd) - Protege - Remote

**Apply:** https://hotfix.jobs/jobs/589b1111-6839-4ada-8182-d97a8d63cf6b
**Canonical:** https://hotfix.jobs/jobs/589b1111-6839-4ada-8182-d97a8d63cf6b