# Member of Technical Staff

**Company:** [Perplexity](https://hotfix.jobs/companies/perplexity)
**Location:** Belgrade, Serbia, London, United Kingdom, Berlin, Germany
**Role:** Data Analytics
**Experience:** 4+ years
**Skills:** SQL, Python, Search Quality, Data Pipelines, Llm-As-A-Judge, Machine Learning, A/B Testing, Statistics, Spark, Databricks
**Posted:** 2026-09-02

> Analyzes and improves search and answer quality by designing metrics, building evaluation and labeling pipelines, and validating changes through experiments. Requires 4+ years of analyst experience, production-grade coding skills, and proficiency in SQL and Python.

## Job Description

## Responsibilities
- Find and diagnose quality issues in the search pipeline.
- Design metrics from scratch to track and measure search quality.
- Build datasets for model training, including LLM-as-a-judge labeling pipelines.
- Improve search snippet quality and page-selection algorithms for indexing.
- Design and analyze A/B experiments to validate improvements.

## Requirements
- 4+ years of experience as a data analyst or in a related role.
- Strong coding skills, including the ability to write production-grade code at a mid-level backend-engineer level.
- Proficiency with SQL and Python.
- Hands-on experience with at least one of the following:
  - Designing metrics from scratch.
  - Building labeling pipelines using LLM-as-a-judge.
  - Training machine-learning models shipped to production with measurable metric improvements.
  - Designing evaluations with known ground truth, such as SimpleQA or BrowseComp, or driving meaningful improvements on such evaluations.

## Preferred Qualifications
- Experience working on search-related products.
- Knowledge of statistics and A/B experiment design.
- Experience with Apache Spark or Databricks.

## Similar jobs

- [Model Risk Management Specialist](https://hotfix.jobs/jobs/33ebef34-dac3-4aca-8d43-1f8cbe4aab96) - Kraken - Remote
- [Alternatives Data Operations Analyst](https://hotfix.jobs/jobs/16a48cad-9163-4057-ab66-6cd008763a42) - Addepar - Edinburgh, United Kingdom
- [Data Analyst — Product Analytics](https://hotfix.jobs/jobs/69894d78-1e75-403c-97eb-02d9dc7d4dca) - Virta Health - Remote - $96k – $125k/yr
- [Data Analyst](https://hotfix.jobs/jobs/7d19cb76-5516-4030-b1ad-2ceb61a0e366) - Supernova Technology - Remote
- [Senior Product Data Analyst](https://hotfix.jobs/jobs/84069a89-960b-4c0c-9f47-10ad5db5e809) - Collectly - Remote

**Apply:** https://hotfix.jobs/jobs/5c297f17-5f6d-445f-9d6c-d21704231e80
**Canonical:** https://hotfix.jobs/jobs/5c297f17-5f6d-445f-9d6c-d21704231e80