Staff Data Scientist - Verification & Validation
Staff Data Scientist designs statistical frameworks and conducts robust analysis to validate safety-critical AI systems for autonomous vehicles. Delivers data-driven insights to engineering teams and leadership, scaling pipelines for petabyte-scale driving data analysis. Requires MS/PhD in quantitative field and expertise in Python, SQL, and statistical methods.
About the job
Responsibilities
- Design evaluation frameworks: Architect statistical methodologies for safety-critical AI systems to form objective, rigorous conclusions about their performance and reliability.
- Conduct robust analysis: Deliver validation evidence to support increasingly complex operations and identify potential edge-case failures.
- Inform strategy: Deliver clear, data-driven insights to development teams to guide system improvement, and to executive leadership to inform milestone-level go/no-go decisions.
- Define metrics: Drive alignment across engineering teams on performance metrics and data extraction strategies.
- Lead the lifecycle: Manage all phases of evaluation including prototyping, requirements capture, design, implementation, and validation.
- Scale pipelines: Partner with engineers to build and maintain scalable data processing and simulation pipelines, applying distributed computing to analyze petabytes of driving data.
Qualifications
- MS or PhD in Statistics, Computer Science, Machine Learning, Applied Mathematics, or related quantitative field
- Proficiency in Python and SQL with experience in production-quality code
- Demonstrated expertise in statistical methodologies including hypothesis testing, power analysis, spatiotemporal modeling, Bayesian inference, and multivariate analysis
- Experience with large-scale data analysis and statistical modeling
- Proficiency with Git, unit testing, and collaborative development practices
Bonus Qualifications
- Hands-on experience with production machine learning pipelines: dataset creation, training frameworks, metrics pipelines
- Experience with modern data processing technologies such as Apache Spark, Spark SQL, and Databricks
- Experience with designing metrics and delivering actionable insights that drive business decisions
Skills
Python, SQL, Git, Bayesian Inference, Hypothesis Testing, Spark, Spark Sql, Databricks, Spatiotemporal Modeling, Multivariate Analysis
Similar jobs
Data Science jobs
Vercel
San Francisco, CALeads the architecture and productionization of advanced consumption forecasting systems supporting financial planning, infrastructure capacity, and executive decisions. Requires staff-level technical leadership, deep time-series and causal modeling expertise, and strong Python and SQL skills.
OpenAI
San Francisco, CAMarketing Scientist supporting OpenAI Ads will design and scale rigorous advertising measurement programs, advise major advertisers, and translate campaign data into actionable performance improvements. The role requires 10+ years of measurement expertise, strong experimentation and econometric skills, and hands-on SQL, Python, or R experience.
Discord
United States
Leads the vision, statistical methodology, research, and education strategy for Discord’s experimentation platform. The role requires a quantitative PhD or equivalent experience, 4+ years in experimentation or causal inference, and proficiency with statistical programming languages.
Mercury
San Francisco, CAStaff Data Scientist focused on building and operating machine learning systems for real-time fraud detection and financial crime risk. The role requires 7+ years working with large datasets, strong Python and SQL skills, production ML experience, and technical leadership.
DuckDuckGo
United StatesStaff Data Scientist driving search ranking and results improvements through machine learning, NLP, evaluation, and ranking-signal development. Requires 10+ years of data science experience, staff-level leadership, production ML deployment, Python, and advanced SQL skills.