# Staff Applied Research Engineer

**Company:** [Drata](https://hotfix.jobs/companies/drata)
**Location:** San Francisco, CA
**Role:** ML Engineering
**Salary:** $221k – $299k/yr
**Experience:** 10+ years
**Skills:** Python, RAG, Information Retrieval, NLP, Embedding Models, Vector Databases, Evaluation Frameworks, A/B Testing, Statistical Analysis, Machine Learning
**Posted:** 2026-05-26

> Drive quality of Drata's AI compliance systems through applied research, focusing on RAG optimization, evaluation frameworks, retrieval strategies, and experiment-driven improvements to document-heavy GenAI workflows.

## Job Description

## What you'll do
- Design and evaluate information access + reasoning strategies across RAG, agents, and classic ML: chunking, embedding models, hybrid search, metadata filtering, structured retrieval, tool use, and multi-step workflows
- Prototype GenAI workflows (including agentic systems) that map and reason over compliance objects (controls ↔ risks ↔ requirements)
- Explore ML + probabilistic approaches where GenAI is not the best fit: classifiers, ranking models, graph/link prediction, calibration, and weak supervision
- Build and maintain evaluation frameworks: golden datasets, automated quality metrics, regression detection
- Implement and tune ranking/reranking systems: cross-encoders, LLM-based rerankers, learning-to-rank, custom scoring functions
- Run experiments to validate hypotheses and quantify improvements before production rollout
- Debug failure modes and build error taxonomies across retrieval, reasoning, and generation
- Collaborate with AI and Software Engineers to hand off validated approaches for productionization
- Stay current on applied research in RAG, agents, LLM evaluation, and relevance modeling; bring innovations into the product

## What you'll bring
- 10+ years of experience in applied research, data science, or ML with a focus on NLP, information retrieval, or knowledge systems
- 2+ years of hands-on experience building or contributing to production AI/ML systems
- Strong foundation in information retrieval: dense and sparse retrieval, embedding models, search relevance
- Experience with RAG systems: chunking strategies, vector databases, retrieval optimization
- Proficiency in evaluation methodology: metrics design, golden dataset creation, A/B testing, statistical analysis
- Strong Python skills and comfort with notebook-driven research workflows
- Experience communicating research findings to engineering teams and translating insights into actionable recommendations
- Bonus: Experience with compliance, legal, or document-heavy domains
- Bonus: Publications or contributions in IR, NLP, or RAG evaluation

## How we support you
- **Shared Success**: Stock equity / RSUs
- **Health & Wellness**: Up to 100% employer-paid medical, dental, vision premiums; wellness benefits; healthcare concierge
- **Financial Well-being**: 401(k), company-paid life/disability insurance, tax-advantaged spending accounts
- **Family Support**: Paid parental leave after 6 months; Kindbody fertility/family-building benefits
- **Growth & Development**: Annual professional/personal development stipends; internal learning opportunities
- **Time Off & Flexibility**: Flexible vacation policy, paid holidays

## Similar roles

- [Staff AI Engineer](https://hotfix.jobs/jobs/01082ecd-aa50-473b-a5f2-3b35f492f607) - Pave - San Francisco, CA - $221k – $299k/yr
- [Member of Technical Staff — Audio and Voice AI](https://hotfix.jobs/jobs/be841ac6-7856-43df-b21f-0540dd7e0f37) - Stuut - San Francisco, CA - $220k – $320k/yr
- [Member of Technical Staff](https://hotfix.jobs/jobs/344874a8-2e4c-4d32-996a-d7003be82d05) - Perplexity - San Francisco, CA - $220k – $405k/yr
- [Member of Technical Staff](https://hotfix.jobs/jobs/a463067e-4ce4-4632-ac39-cbc9264db356) - Perplexity - San Francisco, CA - $220k – $405k/yr
- [Member of Technical Staff](https://hotfix.jobs/jobs/184e2951-ec23-4f0f-ae88-0596e3b8325e) - Perplexity - San Francisco, CA - $220k – $405k/yr

**Apply:** https://hotfix.jobs/jobs/b3470c95-0cf1-4aa5-875c-f7b248efec50
**Canonical:** https://hotfix.jobs/jobs/b3470c95-0cf1-4aa5-875c-f7b248efec50