Skip to content
Scale AIScale AIUnited States

Subject Matter Expert

Part-time hourly Subject Matter Expert performing quality audits, defining evaluation standards and rubrics, analyzing performance gaps, and recommending quality improvements for specialized domain projects. Requires deep expertise in a field such as medicine, law or engineering.

Salary not listed
RemoteOther

About the role

Key Responsibilities

Quality Audits & Evaluation

  • Conduct structured audits of task outputs to evaluate accuracy, consistency, and adherence to project guidelines.
  • Apply domain expertise to assess nuanced or complex work requiring specialized knowledge.
  • Identify patterns of quality issues and flag systemic gaps impacting output standards.

Quality Standards & Benchmarking

  • Define and refine quality benchmarks, scoring rubrics, and evaluation criteria based on project needs.
  • Contribute to the development of gold standards and reference materials for high-quality outputs.
  • Ensure quality frameworks reflect domain-specific rigor where applicable.

Analysis & Insights

  • Analyze audit results to generate actionable insights on quality trends and performance gaps.
  • Provide structured reports with clear recommendations to improve overall output quality.
  • Partner with internal stakeholders to calibrate on quality expectations and scoring consistency.

Process Improvement (Quality-Focused)

  • Recommend improvements to workflows, guidelines, and evaluation processes based on audit findings.
  • Support calibration efforts to ensure alignment across reviewers and quality stakeholders.
  • Contribute to continuous improvement initiatives strictly related to quality outcomes.

Qualifications

  • Deep domain expertise in a specialized field (e.g., medicine, law, engineering, etc.).
  • Strong analytical skills with the ability to evaluate complex outputs against defined standards.
  • Experience in quality assurance, auditing, or evaluation frameworks is preferred.
  • Ability to produce clear, structured, and actionable feedback.
  • High attention to detail and commitment to accuracy and consistency.

Engagement Expectations

  • Hourly, project-based work aligned to audit volume and quality needs.
  • Deliverables include completed audits, calibration input, and quality reports.
  • Operates independently with minimal oversight, focusing on expertise-driven contributions.

Skills

Quality AssuranceAuditingQuality FrameworksBenchmarkingAnalytical SkillsDomain Expertise

Similar roles

Anthropic

Safeguards Enforcement Analyst, Violence & Extremism

AnthropicSan Francisco, CA +2

Safeguards Enforcement Analyst responsible for designing automated enforcement systems, developing evals, reviewing flagged content for violence and extremism risks, and partnering with engineering to mitigate AI misuse that could cause real-world harm. Requires experience in threat intelligence, policy enforcement, or counterterrorism plus SQL proficiency.

285k – 330k/yr
Hybrid5+ YOEOther
Sphere

Indirect Tax Filings Specialist

SphereSan Francisco, CA

Own day-to-day indirect tax filing, preparation, review, and submission across jurisdictions. Drive process automation with Engineering, reconciliations, risk monitoring, and scaling compliance operations in a fast-paced AI-powered tax tech environment.

Salary not listed
Remote4+ YOEOther
Givebutter

SEO Manager

GivebutterNew York, NY +9

SEO Manager partnering with Content Marketing to drive organic growth through keyword research, content optimization, technical collaboration, and experimentation with emerging search channels including AI discovery. Requires 5+ years SEO experience with strong analytical and cross-functional skills.

95k – 110k/yr
Remote5+ YOEOther
Octane

Manager, Credit Risk Strategy

OctaneNew York, NY

Manager overseeing credit risk strategies for captive partner portfolio. Analyze origination/performance data to optimize underwriting, pricing, and profitability while managing risk appetite. Requires 4+ years credit analysis experience, strong Python/SQL skills, and ability to communicate insights to C-level and external partners.

110k – 150k/yr
Hybrid4+ YOEOther
Chainguard

Payroll Analyst, Global Payroll

ChainguardUnited States

Payroll Analyst responsible for building pre-payroll analytics, running end-to-end US payroll cycles, conducting audits, managing Workday inputs/EIBs, and supporting global payroll vendors and compliance as the company scales internationally. Requires 4+ years payroll experience and 3+ years with Workday Payroll.

106k – 125k/yr
Remote4+ YOEOther