# Senior Staff GenAI Engineer - Application Performance Monitoring (APM)

**Company:** [Datadog](https://hotfix.jobs/companies/datadog)
**Location:** New York, NY
**Role:** ML Engineering
**Salary:** $272k – $340k/yr
**Experience:** 10+ years
**Skills:** Generative AI, Machine Learning, Ml Models, Distributed Tracing, Model Deployment, Model Evaluation, Model Optimization, Python, Kubernetes, LLMs
**Posted:** 2026-01-21

> Lead GenAI/ML initiatives for Datadog's APM team, designing and deploying models for agentic workflows, automated investigations, and performance optimization. Requires 10+ years experience with 4+ years leading cross-team projects.

## Job Description

## What You’ll Do:
- Serve as technical owner for GenAI initiatives within APM, leading design, development, and deployment of ML/AI-powered features across multiple teams.
- Guide long-term strategy and technical direction for GenAI workflows across APM and related products.
- Build and benchmark GenAI/ML models using state-of-the-art techniques.
- Contribute to Datadog’s broader senior engineering community through thought leadership and collaboration.
- Collaborate with cross-functional teams to build automated investigation and triaging tools.
- Influence product direction with a strong product mindset, advocating for end users.
- Guide teams through ambiguity, scaling challenges, and evolving requirements.
- Actively mentor engineers and influence engineering culture through design reviews, technical talks, and working groups.

## Who You Are:
- BS/MS/PhD in a scientific field or equivalent experience.
- **10+ years** of relevant engineering experience, including **4+ years** leading cross-team technical initiatives.
- Proven track record of leading large-scale GenAI/ML initiatives in a product-driven environment.
- Highly proficient in model development, deployment, evaluation, and optimization, with hands-on experience in end-to-end ML systems.
- Ability to drive initiatives across cross-functional teams and solve ambiguous challenges.
- Passionate about mentoring and cultivating high-performing engineering culture.
- Excited about leveraging AI tools to enhance coding, problem-solving, and building.

## Benefits and Growth:
- Build tools for software engineers and use them to accelerate development.
- Influence product direction and business impact.
- Work with skilled, knowledgeable teammates.
- Competitive global benefits.
- Continuous professional development.

## Similar roles

- [Member of Technical Staff, Robotics Research Engineer](https://hotfix.jobs/jobs/fe415ab1-7552-44f0-8c2c-2d5ecc12cd8c) - Runway - New York, NY - $270k – $370k/yr
- [Senior Member of Research Staff, Voleon Securities](https://hotfix.jobs/jobs/041f853d-f3fd-46ef-9738-805d0db67119) - The Voleon Group - New York, NY - $275k – $300k/yr
- [Senior Member of Research Staff](https://hotfix.jobs/jobs/40bf0476-d213-4325-8043-a5070e083ece) - The Voleon Group - Berkeley, CA - $275k – $300k/yr
- [Sr. Staff Machine Learning Engineer, Content Quality](https://hotfix.jobs/jobs/8eab56ef-b6cb-4fdf-a991-3b9e1026735e) - Pinterest - San Francisco, CA - $268k – $469k/yr
- [Staff Applied Scientist - Dashboards](https://hotfix.jobs/jobs/513c285d-d6c3-414c-918f-89178ab99304) - Datadog - New York, NY - $276k – $345k/yr

**Apply:** https://hotfix.jobs/jobs/7152623c-7d76-46ff-b04e-66178ff00e97
**Canonical:** https://hotfix.jobs/jobs/7152623c-7d76-46ff-b04e-66178ff00e97