What You'll Do
Evaluation, Measurement, and Empirical Analysis
- Conduct quantitative evaluations of Abridge models and products using data from real-world deployments, offline evaluations, and customer feedback
- Develop and validate metrics that reflect meaningful outcomes for providers and patients — including assessing construct validity, characterizing measurement error, and surfacing selection bias in observational signals like user ratings and feedback
- Design and execute analyses that address the team's core research questions: validating automated evaluation frameworks against human judgment, characterizing heterogeneity in adoption and usage trajectories, estimating causal effects of ambient AI on clinical and operational outcomes, and extracting structured characterizations of clinical practice from unstructured conversation data
- Build deep familiarity with existing metrics and evaluation frameworks used at Abridge and beyond, interrogating underlying assumptions and proposing alternatives where appropriate
Data Infrastructure and Expertise
- Develop and maintain deep expertise in Abridge's data assets — including production data, user feedback signals, and clinical conversation data — and serve as the team's authority on data provenance, structure, and limitations for research studies
- Build, extend, and maintain the data pipelines that support internal and external research efforts, working across raw data sources to produce clean, well-documented, research-ready datasets
- Collaborate with Data Engineering and platform teams to ensure that the data the research team needs is accessible, reliable, and well-understood
Cross-Functional Research Collaboration and Communication
- Collaborate closely with product, engineering, science, and data teams to ensure evaluation and analysis are credible, decision-relevant, and grounded in a deep understanding of product development and integration
- Partner with commercial teams, and liaise with customers through relationships owned by our Partner Experience organization, to ensure measurement and evaluation reflect how products are used and experienced in real-world practice
- Translate complex analyses into clear, nuanced narratives grounded in data, tailoring communication to different audiences and contexts
- Produce technical analyses, reports, and presentations that inform product decisions, guide strategy, and contribute to a rigorous evidence base for the real-world impact of ambient AI in healthcare
What You'll Bring
- 8+ years using SQL and Python or R for data science, including experience building or working closely with data pipelines and data infrastructure
- 3+ years of data science experience in academic or industry research settings where you contributed to research studies relying on complex data processing and analysis
- Comfort operating across the spectrum from data engineering to applied science — you can develop data pipelines and also conduct a rigorous evaluation study in collaboration with research scientists
- Experience with code-based data visualization tools (Seaborn, ggplot2)
- Demonstrated ability to conduct empirical evaluations using observational or experimental data, grounded in a rigorous quantitative or mixed-methods approach
- A problem-before-method mindset: you do not change the question to make it amenable to simple analysis, but instead push the methodological frontier to solve the real-world problems that matter to health systems, clinicians, and patients
- Excellent communicator capable of effectively delivering quantitative findings to non-technical stakeholders in a clear and compelling fashion
- A team player, comfortable assisting others across the organization in solving data problems and answering questions with data
- Must be willing to work from our NYC office at least 3x per week
This position requires a commitment to a hybrid work model, with the expectation of coming into the office a minimum of (3) three times per week. Relocation assistance is available for candidates willing to move to New York City.