# Member of Technical Staff, Data Analysis and Evaluation

**Company:** [Cohere](https://hotfix.jobs/companies/cohere)
**Location:** Remote
**Role:** Data Science
**Skills:** Python, PyTorch, TensorFlow, JAX, LLMs, Machine Learning, Statistical Analysis, Experimental Design, Data Collection, Distributed Training, Dataset Evaluation, Human Annotation
**Posted:** 2025-12-15

> This role designs data collection and evaluation processes, analyzes dataset quality and model robustness, and trains large language models on distributed infrastructure. It requires strong software engineering, statistics, experimental design, machine learning, and cross-functional communication skills.

## Job Description

## Responsibilities
- Design and oversee data collection tasks, including supporting human annotators and ensuring data quality.
- Develop and apply statistical methods to evaluate dataset quality and reliability.
- Analyze and assess the generalizability and robustness of machine learning systems across diverse use cases.
- Collaborate with cross-functional teams to improve dataset quality and model performance.
- Train and fine-tune large language models (LLMs) on distributed training infrastructure.
- Conduct experiments to evaluate model performance and identify areas for improvement.

## Requirements
- Extremely strong software engineering skills.
- Strong expertise designing and conducting data collection tasks, including working with human annotators.
- Strong statistical skills and experience evaluating scientific experiments related to data collection and model performance.
- Experience analyzing datasets for quality, bias, and suitability for training machine learning models.
- Hands-on experience training LLMs on distributed training infrastructure.
- Familiarity with evaluating and improving the generalizability and robustness of machine learning systems.
- Proficiency in Python and machine learning frameworks such as PyTorch, TensorFlow, or JAX.
- Excellent communication skills for collaborating with cross-functional teams and presenting findings.
- One or more papers at top-tier venues such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, or EMNLP.

## Compensation and Benefits
- Weekly lunch stipend of $75/£75 or equivalent in local currency.
- Full health and dental benefits, including a separate mental health budget.
- RRSP matching, 401(k), or pension scheme.
- 100% parental leave top-up for up to six months for either parent.
- Annual enrichment benefits covering arts and culture, fitness and wellness, quality time, and workspace improvements.
- Education and learning stipend for conferences, courses, and coaching.
- Six weeks of paid vacation (30 working days).
- Travel budget for remote employees to visit other offices and an annual company offsite.
- Coworking benefit for employees who are not near an office.
- $500 home office stipend.

## Similar jobs

- [Data Scientist - Perception Verification and Validation](https://hotfix.jobs/jobs/8338c83e-9d2e-41a2-be60-edd8518d148e) - Zoox - Boston, MA - $167k – $228k/yr
- [Data Scientist](https://hotfix.jobs/jobs/eec79072-5f55-4c28-9f70-4c336fe7d7d7) - Dataiku - London, United Kingdom
- [Scientist](https://hotfix.jobs/jobs/48a97d2c-5ac5-478e-b1ed-930ef6fe673f) - Axle - Bethesda, MD - $120k – $125k/yr
- [Data Scientist, Fraud Risk](https://hotfix.jobs/jobs/4ca84840-51ff-4fae-bf40-96d37028a850) - Imprint - Remote - $170k – $200k/yr
- [Data Science Manager - Market Origination](https://hotfix.jobs/jobs/85839332-0cf4-472c-ba19-87d58e1c3c97) - PrizePicks - Remote - $160k – $190k/yr

**Apply:** https://hotfix.jobs/jobs/10a440c0-1064-4429-a883-5fcbfc954428
**Canonical:** https://hotfix.jobs/jobs/10a440c0-1064-4429-a883-5fcbfc954428