Skip to content
DatadogDatadogNew York, NY

Staff Applied Scientist - Dashboards

Staff Applied Scientist defining evaluation strategy and quality metrics for Datadog's AI-native Dashboards product. Owns ML/GenAI evaluation systems, builds datasets and harnesses, and drives improvements in retrieval, tool selection, and agent performance.

276k – 345k
Hybrid10+ YOEML Engineering

About the role

What You’ll Do

  • Own the evaluation strategy for Dashboards, as well as sister teams within our organization. Define the metrics — offline and online, quality and cost, single-turn and multi-turn — that the team and the broader organization optimize against.
  • Build the eval datasets, golden traces, and regression harnesses that catch quality changes before they hit customers, and make those assets reusable by every team that is building dashboards and widgets through agents.
  • Drive measurable improvements to retrieval relevance, tool-selection accuracy, and context efficiency, partnering closely with the engineers on the team.
  • Provide technical leadership across the Dashboards team and the broader organization through design reviews, working groups, and mentorship.

Who You Are

  • BS/MS/PhD in a scientific field, or equivalent experience.
  • 10+ years of relevant engineering or applied science experience, including time as a technical lead.
  • Proven track record of leading ML or GenAI initiatives in a product-driven environment, from research through production.
  • Significant experience with evaluation, experimentation, or measurement of ML systems at scale.
  • Strong product mindset and comfortable driving initiatives across cross-functional teams.
  • Thrive in ambiguity and can make sound technical calls when the path isn’t yet defined.

Skills

Machine LearningGenerative AIEvaluation FrameworksExperimentationRetrieval SystemsTool SelectionContext EfficiencyRegression TestingTechnical LeadershipCross-Functional Collaboration

Similar roles

ML Engineering jobs
Square

Staff Machine Learning Engineer, Underwriting and Credit

SquareUnited States

Senior IC building and maintaining ML underwriting and credit decisioning models for Cash App Borrow and Afterpay. Owns full modeling lifecycle including experimentation, calibration, deployment, and monitoring.

277k – 415k
Remote10+ YOEML Engineering
Square

Staff Applied Machine Learning Engineer - Fraud & Abuse

SquareSan Francisco, CA

Staff Applied ML Engineer building and operating production ML decision systems to detect and prevent payment fraud, scams, identity abuse, and marketplace risk across Block.

277k – 415k
On-site12+ YOEML Engineering
Square

Staff Applied Machine Learning Engineer

SquareUnited States

Build and operate production ML systems for ranking, recommendations, search, and customer intelligence signals used across product, growth, risk, and decisioning teams. Requires 12+ years of production ML experience and deep expertise in intelligent systems.

277k – 415k
Remote12+ YOEML Engineering
The Voleon Group

Senior Member of Research Staff, Voleon Securities

The Voleon GroupNew York, NY +1

Develops and optimizes AI/ML models for financial market prediction and portfolio optimization in securities trading. Requires 5+ years directing research, PhD-level expertise in ML/statistics, and production coding skills.

275k – 300k
Hybrid5+ YOEML Engineering
The Voleon Group

Senior Member of Research Staff

The Voleon GroupBerkeley, CA

Leads key research projects in statistical machine learning for financial market prediction and portfolio optimization, from basic research to production deployment. Requires 5+ years experience, PhD-level expertise, strong ML/math skills, and Python/R proficiency.

275k – 300k
Hybrid5+ YOEML Engineering