Staff Engineer – Experimentation Team
Builds statistical engines and warehouse-native analysis pipelines for A/B testing and adaptive experimentation (contextual bandits). Requires 10+ years experience, deep applied statistics/ML, backend expertise in Go/Python, and warehouse integration across Snowflake, Databricks, etc.
Responsibilities
- Build the experimentation statistical engine — hypothesis testing, sequential analysis, variance reduction (CUPED, Winsorization), power analysis. Ensure statistical correctness across all experiment types.
- Design warehouse-native experimentation that runs analysis inside customer warehouses (Snowflake, Databricks, Redshift, BigQuery). Build modular, warehouse-agnostic abstractions for rapid new backend support.
- Lead adaptive experimentation — contextual bandit systems, Bayesian optimization, automated allocation beyond simple A/B tests.
- Drive the platform roadmap with product, design, and data science. Shape what we build, not just how.
- Collaborate cross-functionally with Warehouse Integrations, SDK, Platform, and Data Science teams.
- Mentor engineers and raise the team's bar for statistical rigor and system design.
- Own operational excellence — monitoring, observability, incident response, on-call. Robust telemetry and alerting.
Qualifications
- 10+ years building large-scale experimentation platforms, statistical analysis systems, or data-intensive backend services.
- Applied-statistics knowledge: hypothesis testing, sequential analysis, variance reduction (CUPED), power analysis, experiment design. Comfortable with frequentist vs. Bayesian trade-offs.
- Experience with adaptive experimentation ML — contextual bandits, Thompson sampling, Bayesian optimization, or RL-based allocation.
- Track record designing warehouse-agnostic systems across Snowflake, Databricks, Redshift, BigQuery, or similar.
- Expertise in Go, Python, or similar for backend services and statistical computation.
- Experience with event-driven architectures, data pipelines, and large-scale data processing.
- Cloud environments (AWS, GCP) with infrastructure-as-code.
- Technical leadership: setting direction, breaking down complex problems, influencing across teams.
- Ability to translate statistical concepts for product and engineering audiences.
Pay
Target pay ranges based on Geographic Zones for Level 5:
Zone 1 (San Francisco/Bay Area or NYC Metropolitan Area, Boston, Seattle): $214,800 - $295,350
Zone 2 (Irvine, LA, Monterey, Santa Barbara, Santa Rosa, Austin, Portland, Philadelphia, Chicago): $193,400 - $265,870
Zone 3 (All other US locations): $182,600 - $251,020
Exact compensation may vary based on skills, experience, and location. Includes RSUs, health, vision, dental insurance, and mental health benefits.