# Member of Technical Staff

**Company:** [Perplexity](https://hotfix.jobs/companies/perplexity)
**Location:** San Francisco, CA
**Role:** Data Science
**Salary:** $200k – $300k/yr
**Experience:** 4+ years
**Skills:** Python, SQL, AWS, Databricks, LLMs, Vlm, Machine Learning, Data Science, Llm-As-A-Judge
**Posted:** 2026-06-29

> Build specialized evals and automated pipelines to measure and improve answer quality for Perplexity's LLM-powered search engine, focusing on retrieval, tool calls, and visual rendering. Requires 4+ years in data science/ML, strong Python/SQL, and cloud experience (MS/PhD preferred).

## Job Description

## Responsibilities
- Architect and maintain automated evaluation pipelines to assess answer quality across Perplexity's products, ensuring high standards for accuracy and helpfulness.
- Design evaluation sets and methods specifically to measure the impact of tool calls (particularly web search retrieval) on the final answer's quality.
- Develop VLM-based solutions to programmatically evaluate how final answers render visually across different platforms and devices.
- Continuously review public benchmarks and academic evaluations for their applicability to the Perplexity product, adapting and incorporating them into our regular performance measurements.
- Operate within a small, high-impact team where your evaluation metrics directly shape product changes, collaborating closely with technical leadership to measure and improve Answer Quality.

## Requirements
- PhD or MS in a technical field or equivalent experience.
- 4+ years of experience in data science or machine learning.
- Strong proficiency in Python and SQL (expected to write production-grade code).
- Experience building within a modern cloud data stack, specifically AWS and Databricks.
- Comfortable with agentic coding workflows and using AI-assisted development tools to iterate faster.

## Preferred Qualifications
- 1+ years of experience working with LLMs at scale, specifically with LLM-as-a-judge setups.
- Prior experience working on customer-facing web products or consumer apps, with real user traffic at scale.
- A strong research background, with experience applying research methods to real-world ML problems.
- Experience defining evaluation metrics (e.g., factual consistency, hallucination rate, retrieval precision) and building ground truth datasets.

## Similar roles

- [Staff Data Scientist](https://hotfix.jobs/jobs/cfc20140-a199-4647-983d-94cb66dcc646) - Airbnb - Remote - $200k – $242k/yr
- [Staff Data Scientist, Marketplace](https://hotfix.jobs/jobs/b4332cf1-d467-452e-8c7b-daeaceadef43) - Zocdoc - New York, NY - $200k – $270k/yr
- [Staff Data Scientist](https://hotfix.jobs/jobs/e4b33b39-f3ec-4bb1-8491-6b78aba213f6) - Imprint - New York, NY - $200k – $250k/yr
- [Staff Data Scientist](https://hotfix.jobs/jobs/13cce944-77d8-478c-8aba-cbf8d9481169) - OpenX - Remote - $196k – $219k/yr
- [Staff Data Scientist](https://hotfix.jobs/jobs/d37488d2-38c6-47fe-84dc-5609c4e3ec2b) - Sift - Remote - $195k – $265k/yr

**Apply:** https://hotfix.jobs/jobs/b4a2e18f-c485-4bd7-ae16-2834ff84152d
**Canonical:** https://hotfix.jobs/jobs/b4a2e18f-c485-4bd7-ae16-2834ff84152d