Staff Data Scientist

Responsibilities

• Define the end-to-end methodology every team follows - hypothesis → metrics → design → power → readout → decision - and make it the default
• Own the statistical approach (significance, multiple comparisons, sequential testing, variance reduction like CUPED) for small-sample, fast-paced contexts where classic A/B power is hard to reach
• Build the methods toolkit for our clustered, hierarchical data (user → sub-account/location → agency), where randomization and analysis units differ
• Apply rigorous causal inference (matching, diff-in-diff, instrumental variables, synthetic control, etc) when clean experiments aren't feasible - churn, onboarding, GTM - separating real signal from selection bias, seasonality, and mix effects
• Own the design discipline for running many experiments at once - layering, orthogonal experiments, holdouts, and guardrails that keep concurrent tests from contaminating each other
• Partner with AI/ML teams to design and evaluate experiments for AI features, including measurement for non-deterministic, fast-iterating systems
• Run the experiment review forum and hold the line on what counts as a real result
• Build the Experimentation curriculum and templates that level up PMs and analysts so good design scales beyond you
• Partner with Analytics Engineering on governed, experiment-ready data and consistent metric definitions
• Influence leadership and cross-functional partners on where to invest, translating statistical nuance into clear, decision-grade guidance

Requirements

• 9+ years in data science, product analytics, or applied statistics, with deep hands-on experience designing and analyzing online controlled experiments at scale
• Strong applied statistics - frequentist foundations, Bayesian methods, power analysis, variance reduction, and the failure modes of A/B testing (peeking, multiple testing, network/cluster effects)
• Practical causal inference, with sound judgment about when a result is causal versus an artifact of how the data was generated
• Experience in small-sample, fast-paced, multi-product environments -you know when a decision needs a clean experiment and when it needs a fast, good-enough read
• Strong SQL and working proficiency in Python or R
• Cross-functional and senior-leadership influence - you raise others' experiment quality without direct authority

Nice to Have

• Familiarity with a modern experimentation platform such as Statsig
• Experience building an experimentation practice or culture from the ground up
• Background in B2B SaaS, CRM, or product-led growth, and familiarity with the measurement challenges these motions create
• Multi-tenant or marketplace product experience (agency → sub-account → end-customer structures)