# Staff Applied Scientist - Dashboards

**Company:** [Datadog](https://hotfix.jobs/companies/datadog)
**Location:** New York, NY
**Role:** ML Engineering
**Salary:** $276k – $345k/yr
**Experience:** 10+ years
**Skills:** Machine Learning, Generative AI, Evaluation Frameworks, Experimentation, Retrieval Systems, Tool Selection, Context Efficiency, Regression Testing, Technical Leadership, Cross-Functional Collaboration
**Posted:** 2026-06-05

> Staff Applied Scientist defining evaluation strategy and quality metrics for Datadog's AI-native Dashboards product. Owns ML/GenAI evaluation systems, builds datasets and harnesses, and drives improvements in retrieval, tool selection, and agent performance.

## Job Description

## What You’ll Do

- Own the evaluation strategy for Dashboards, as well as sister teams within our organization. Define the metrics — offline and online, quality and cost, single-turn and multi-turn — that the team and the broader organization optimize against.
- Build the eval datasets, golden traces, and regression harnesses that catch quality changes before they hit customers, and make those assets reusable by every team that is building dashboards and widgets through agents.
- Drive measurable improvements to retrieval relevance, tool-selection accuracy, and context efficiency, partnering closely with the engineers on the team.
- Provide technical leadership across the Dashboards team and the broader organization through design reviews, working groups, and mentorship.

## Who You Are

- BS/MS/PhD in a scientific field, or equivalent experience.
- 10+ years of relevant engineering or applied science experience, including time as a technical lead.
- Proven track record of leading ML or GenAI initiatives in a product-driven environment, from research through production.
- Significant experience with evaluation, experimentation, or measurement of ML systems at scale.
- Strong product mindset and comfortable driving initiatives across cross-functional teams.
- Thrive in ambiguity and can make sound technical calls when the path isn’t yet defined.

## Similar roles

- [Staff Machine Learning Engineer, Underwriting and Credit](https://hotfix.jobs/jobs/b61761e2-2aef-46fd-888b-9f738b8c47f2) - Square - Remote - $277k – $415k/yr
- [Staff Applied Machine Learning Engineer - Fraud & Abuse](https://hotfix.jobs/jobs/c8ea2c50-bd8e-410a-9116-6abf45a49a47) - Square - San Francisco, CA - $277k – $415k/yr
- [Staff Applied Machine Learning Engineer](https://hotfix.jobs/jobs/e1528eb9-6d81-4ecf-b9d5-ebc31ee66a75) - Square - Remote - $277k – $415k/yr
- [Senior Member of Research Staff, Voleon Securities](https://hotfix.jobs/jobs/041f853d-f3fd-46ef-9738-805d0db67119) - The Voleon Group - New York, NY - $275k – $300k/yr
- [Senior Member of Research Staff](https://hotfix.jobs/jobs/40bf0476-d213-4325-8043-a5070e083ece) - The Voleon Group - Berkeley, CA - $275k – $300k/yr

**Apply:** https://hotfix.jobs/jobs/513c285d-d6c3-414c-918f-89178ab99304
**Canonical:** https://hotfix.jobs/jobs/513c285d-d6c3-414c-918f-89178ab99304