Agent Data Scientist
Senior individual contributor responsible for defining, operationalizing, and improving performance metrics for enterprise AI agent deployments. The role combines customer-facing advisory work, quantitative analysis, experimentation, and building scalable measurement tools and frameworks.
About the job
Responsibilities
- Work directly with customers to define performance metrics and operationalize them for trusted use.
- Own metric delivery, transparent data access, and performance reporting.
- Diagnose performance drivers, design experiments, and drive improvements on customer accounts.
- Build product features, tools, and frameworks that help Decagon scale measurement practices.
- Define success measures for new initiatives without established precedents.
- Provide feedback to Product, Engineering, and Research to shape the product roadmap.
Requirements
- 5+ years in data science, analytics, or another quantitative role, driving analyses end to end and influencing decisions.
- Exceptional judgment in interpreting metrics and prioritizing high-impact analysis.
- Comfort working in fast-moving, ambiguous environments.
- Strong independence and ability to set priorities against a broad mandate.
- Ability to align stakeholders with differing priorities and technical expertise.
- Fluency in SQL and Python.
- Comfort working with large volumes of messy, unstructured conversational data.
Nice-to-haves
- Experience building internal tooling with meaningful adoption.
- Experience working on AI products in production, including evaluating LLM or agent behavior and building evaluations.
Compensation & Benefits
- $165K-$215K + equity
- Medical, dental, and vision benefits for employees and families.
- Life insurance and disability benefits.
- Retirement plan.
- Parental leave.
- Fertility and family-building benefits through Carrot.
- Monthly wellness and lifestyle stipend.
- Daily office lunches and snacks.
- Take-what-you-need vacation policy.
Skills
SQL, Python, Data Science, Data Analytics, A/B Testing, Metric Design, Data Visualization, Large-Scale Data, Conversational Data, Llm Evaluation, AI Agents, Internal Tooling
Similar jobs
Data Science jobs
Rula
United States
The Data Scientist will analyze healthcare claims, billing patterns, and complex datasets to identify anomalies and support Trust & Safety, compliance, and legal operations. The role requires 5+ years of experience, expert SQL and Python/R skills, data visualization expertise, and strong risk-focused communication.
Zoox
Boston, MA
Develop datasets, performance metrics, and statistical methods to verify and validate an autonomous driving perception stack. The role requires advanced education in a quantitative field and proficiency in Python, SQL or Spark/Scala, exploratory analysis, visualization, and statistical modeling.
Imprint
United States
Owns data science and decisioning for onboarding fraud, identity, and KYC, developing predictive models, experiments, vendor evaluations, and monitoring systems. Requires 5+ years in data science or risk analytics, strong Python and SQL, and experience with adversarial classification problems.
PrizePicks
United States
Leads the data science function responsible for sports and esports market origination, projection accuracy, and coverage strategy. Requires 5+ years in sports betting, fantasy, or gaming, strong modeling skills in SQL and Python or R, and experience scaling high-performing teams.
Haus
San Francisco, CAOwn customer-facing causal marketing measurement engagements, from model configuration and QA through interpretation, recommendations, escalations, and enablement. The role requires direct MMM experience, marketing experimentation knowledge, and strong customer communication skills.