# Staff GenAI Engineer - Application Performance Monitoring

**Company:** [Datadog](https://hotfix.jobs/companies/datadog)
**Location:** New York, NY
**Role:** ML Engineering
**Salary:** $234k – $234k/yr
**Experience:** 10+ years
**Skills:** Generative AI, Machine Learning, Model Deployment, Model Training, Fine-Tuning, Model Evaluation, Benchmarking, Technical Leadership
**Posted:** 2026-06-30

> Technical leader on Datadog's APM team building and deploying GenAI/ML models for agentic investigations, automated troubleshooting, and incident triaging. Requires 10+ years experience leading large-scale GenAI initiatives end-to-end in product environments.

## Job Description

## What You’ll Do

- Act as a technical leader within the APM organization, driving GenAI/machine learning projects from concept to production.
- Build and benchmark GenAI/ML models using state-of-the-art techniques.
- Collaborate with cross-functional teams to build automated investigation and triaging tools.
- Influence product direction by bringing a strong product mindset to your work, always advocating for the end user.
- Guide teams through ambiguity, scaling challenges, and evolving requirements with clear technical direction.
- Actively mentor engineers and influence engineering culture through leadership in design reviews, technical talks, and working groups.

## Who You Are

- BS/MS/PhD in a scientific field or equivalent experience.
- 10+ years of relevant engineering experience, as well as experience acting as a technical lead.
- Proven track record of leading large-scale GenAI/ML initiatives in a product-driven environment.
- Significant experience in model deployment, development, training, fine-tuning, or evaluation.
- Ability to drive initiatives across cross-functional teams, and solve ambiguous challenges.

## Similar roles

- [Staff Software Engineer - ML Observability](https://hotfix.jobs/jobs/29e1a9d6-f50c-4626-8874-48c117a5c52e) - Datadog - Boston, MA - $234k – $300k/yr
- [Staff AI Engineer - Cortex Code Quality](https://hotfix.jobs/jobs/1274746b-863d-4937-84f2-c41ac854c1b9) - Snowflake - Menlo Park, CA - $236k – $339k/yr
- [Staff Software Engineer, Agents](https://hotfix.jobs/jobs/a97dd7fb-bd69-4449-8010-971ab051bbfa) - Harvey - San Francisco, CA - $231k – $340k/yr
- [Senior Staff Software Engineer, AI Model LifeCycle](https://hotfix.jobs/jobs/75876f8d-3aba-4f82-9049-17169f5eec63) - Crusoe - San Francisco, CA - $238k – $318k/yr
- [Staff Machine Learning Engineer](https://hotfix.jobs/jobs/ebc73812-e806-404d-9f4e-435da967dbf5) - Reddit - Remote - $230k – $322k/yr

**Apply:** https://hotfix.jobs/jobs/bea56c44-c929-42b7-a6be-897d60cf826f
**Canonical:** https://hotfix.jobs/jobs/bea56c44-c929-42b7-a6be-897d60cf826f