# Senior Applied Research Engineer

**Company:** [Drata](https://hotfix.jobs/companies/drata)
**Location:** San Francisco, CA
**Role:** ML Engineering
**Salary:** $167k – $226k/yr
**Experience:** 5+ years
**Skills:** Python, RAG, Information Retrieval, NLP, Machine Learning, Vector Databases, Evaluation Metrics, A/B Testing, Statistical Analysis, Embedding Models
**Posted:** 2026-05-26

> Senior Applied Research Engineer driving AI system quality through experimentation and evaluation of RAG, retrieval, and reasoning systems. Requires 5+ years applied ML/NLP experience with strong Python and evaluation methodology skills.

## Job Description

## Responsibilities
- Design and evaluate information access + reasoning strategies across RAG, agents, and classic ML: chunking, embedding models, hybrid search, metadata filtering, semantic routing
- Prototype GenAI workflows (including agentic systems) that map and reason over compliance objects (controls ↔ risks ↔ requirements ↔ evidence)
- Explore ML + probabilistic approaches where GenAI is not the best fit: classifiers, ranking models, graph/link prediction, calibration, and structured prediction
- Build and maintain evaluation frameworks: golden datasets, automated quality metrics, regression detection
- Implement and tune ranking/reranking systems: cross-encoders, LLM-based rerankers, learning-to-rank, custom scoring functions
- Run experiments to validate hypotheses and quantify improvements before production rollout
- Debug failure modes and build error taxonomies across retrieval, reasoning, and generation
- Collaborate with AI and Software Engineers to hand off validated approaches for productionization
- Stay current on applied research in RAG, agents, LLM evaluation, and relevance modeling; bring innovations into the product

## Requirements
- 5+ years of experience in applied research, data science, or ML with a focus on NLP, information retrieval, or knowledge systems
- 2+ years of hands-on experience building or contributing to production AI/ML systems
- Strong foundation in information retrieval: dense and sparse retrieval, embedding models, search relevance
- Experience with RAG systems: chunking strategies, vector databases, retrieval optimization
- Proficiency in evaluation methodology: metrics design, golden dataset creation, A/B testing, statistical significance
- Strong Python skills and comfort with notebook-driven research workflows
- Experience communicating research findings to engineering teams and translating insights into actionable improvements

## Nice-to-Haves
- Experience with compliance, legal, or document-heavy domains
- Publications or contributions in IR, NLP, or RAG evaluation

## Compensation & Benefits
- Competitive base salary: $166,900 - $225,900
- Stock equity (RSUs)
- Up to 100% employer-paid medical, dental, and vision premiums
- 401(k) plan, company-paid life and disability insurance
- Paid Parental Leave (after 6 months)
- Kindbody fertility and family-building benefits
- Generous annual professional and personal development stipends
- Flexible vacation policy and paid holidays

## Similar roles

- [Senior AI Engineer, Agent Harness](https://hotfix.jobs/jobs/c5e7d0e4-19a7-4938-aa4c-9ef179acdd19) - Drata - San Francisco, CA - $167k – $226k/yr
- [Senior Machine Learning Operations Engineer](https://hotfix.jobs/jobs/d3b3a89a-90b9-4f8c-995d-168d4de2ff75) - Mercury - San Francisco, CA - $167k – $208k/yr
- [Senior Software Engineer, Model Serving](https://hotfix.jobs/jobs/b9f85503-574d-49d4-aa09-a2560df24787) - Databricks - San Francisco, CA - $166k – $225k/yr
- [Senior Machine Learning Engineer - GenAI Platform](https://hotfix.jobs/jobs/56b930a9-1d7c-427a-a3df-a38a7a5021f6) - Databricks - San Francisco, CA - $166k – $225k/yr
- [Senior Software Engineer, AI Infrastructure](https://hotfix.jobs/jobs/dd1c3420-6c0a-47ed-a1a7-de401bdf506c) - Ambient.ai - Redwood City, CA - $168k – $205k/yr

**Apply:** https://hotfix.jobs/jobs/9273299c-3014-4320-98e1-ae4b9d661aad
**Canonical:** https://hotfix.jobs/jobs/9273299c-3014-4320-98e1-ae4b9d661aad