Skip to content
DatadogDatadogNew York, NY

Staff GenAI Engineer - Application Performance Monitoring

Technical leader on Datadog's APM team building and deploying GenAI/ML models for agentic investigations, automated troubleshooting, and incident triaging. Requires 10+ years experience leading large-scale GenAI initiatives end-to-end in product environments.

234k – 234k/yr
Hybrid10+ YOEML Engineering

About the role

What You’ll Do

  • Act as a technical leader within the APM organization, driving GenAI/machine learning projects from concept to production.
  • Build and benchmark GenAI/ML models using state-of-the-art techniques.
  • Collaborate with cross-functional teams to build automated investigation and triaging tools.
  • Influence product direction by bringing a strong product mindset to your work, always advocating for the end user.
  • Guide teams through ambiguity, scaling challenges, and evolving requirements with clear technical direction.
  • Actively mentor engineers and influence engineering culture through leadership in design reviews, technical talks, and working groups.

Who You Are

  • BS/MS/PhD in a scientific field or equivalent experience.
  • 10+ years of relevant engineering experience, as well as experience acting as a technical lead.
  • Proven track record of leading large-scale GenAI/ML initiatives in a product-driven environment.
  • Significant experience in model deployment, development, training, fine-tuning, or evaluation.
  • Ability to drive initiatives across cross-functional teams, and solve ambiguous challenges.

Skills

Generative AIMachine LearningModel DeploymentModel TrainingFine-TuningModel EvaluationBenchmarkingTechnical Leadership

Similar roles

ML Engineering jobs
Datadog

Staff Software Engineer - ML Observability

DatadogBoston, MA +1

Lead development of LLM observability features at Datadog, building tools for monitoring, tracing, and evaluating AI systems in production. Requires expertise in distributed systems, GenAI applications, and model internals.

234k – 300k/yr
HybridML Engineering
Snowflake

Staff AI Engineer - Cortex Code Quality

SnowflakeMenlo Park, CA

Staff AI Engineer builds and owns quality systems for Cortex Code AI coding agents, including agent strategy, experimentation pipelines, failure analysis, and cross-team alignment. Requires 8+ years shipping AI/ML software, proficiency in Python/TypeScript/Go, and expertise in LLM evaluation harnesses.

236k – 339k/yr
On-site8+ YOEML Engineering
Harvey

Staff Software Engineer, Agents

HarveySan Francisco, CA +1

Builds AI agent systems for legal workflows, optimizing performance via prompt engineering, model selection, tools, and evaluations. Partners with customers/PMs to ship low-latency agents using Python and LLM APIs; requires 8+ years experience.

231k – 340k/yr
Hybrid8+ YOEML Engineering
Crusoe

Senior Staff Software Engineer, AI Model LifeCycle

CrusoeSan Francisco, CA +1

Builds and manages platforms for AI model lifecycles, focusing on fine-tuning, training pipelines, and reinforcement learning for LLMs. Requires 8+ years in AI, advanced degree, and hands-on experience with generative AI techniques.

238k – 318k/yr
On-site8+ YOEML Engineering
Reddit

Staff Machine Learning Engineer

RedditSan Francisco, CA

Staff Machine Learning Engineer designing, training, and deploying advanced ML models (DNNs, transformers, RL) for Reddit's large-scale online advertising ranking, optimization, and marketplace systems. Requires Master's, 3+ years experience, technical leadership, and expertise in production ML systems.

230k – 322k/yr
Remote7+ YOEML Engineering