Skip to content
DrataDrataSan Francisco, CA

Staff Applied Research Engineer

Drive quality of Drata's AI compliance systems through applied research, focusing on RAG optimization, evaluation frameworks, retrieval strategies, and experiment-driven improvements to document-heavy GenAI workflows.

221k – 299k/yr
Hybrid10+ YOEML Engineering

About the role

What you'll do

  • Design and evaluate information access + reasoning strategies across RAG, agents, and classic ML: chunking, embedding models, hybrid search, metadata filtering, structured retrieval, tool use, and multi-step workflows
  • Prototype GenAI workflows (including agentic systems) that map and reason over compliance objects (controls ↔ risks ↔ requirements)
  • Explore ML + probabilistic approaches where GenAI is not the best fit: classifiers, ranking models, graph/link prediction, calibration, and weak supervision
  • Build and maintain evaluation frameworks: golden datasets, automated quality metrics, regression detection
  • Implement and tune ranking/reranking systems: cross-encoders, LLM-based rerankers, learning-to-rank, custom scoring functions
  • Run experiments to validate hypotheses and quantify improvements before production rollout
  • Debug failure modes and build error taxonomies across retrieval, reasoning, and generation
  • Collaborate with AI and Software Engineers to hand off validated approaches for productionization
  • Stay current on applied research in RAG, agents, LLM evaluation, and relevance modeling; bring innovations into the product

What you'll bring

  • 10+ years of experience in applied research, data science, or ML with a focus on NLP, information retrieval, or knowledge systems
  • 2+ years of hands-on experience building or contributing to production AI/ML systems
  • Strong foundation in information retrieval: dense and sparse retrieval, embedding models, search relevance
  • Experience with RAG systems: chunking strategies, vector databases, retrieval optimization
  • Proficiency in evaluation methodology: metrics design, golden dataset creation, A/B testing, statistical analysis
  • Strong Python skills and comfort with notebook-driven research workflows
  • Experience communicating research findings to engineering teams and translating insights into actionable recommendations
  • Bonus: Experience with compliance, legal, or document-heavy domains
  • Bonus: Publications or contributions in IR, NLP, or RAG evaluation

How we support you

  • Shared Success: Stock equity / RSUs
  • Health & Wellness: Up to 100% employer-paid medical, dental, vision premiums; wellness benefits; healthcare concierge
  • Financial Well-being: 401(k), company-paid life/disability insurance, tax-advantaged spending accounts
  • Family Support: Paid parental leave after 6 months; Kindbody fertility/family-building benefits
  • Growth & Development: Annual professional/personal development stipends; internal learning opportunities
  • Time Off & Flexibility: Flexible vacation policy, paid holidays

Skills

PythonRAGInformation RetrievalNLPEmbedding ModelsVector DatabasesEvaluation FrameworksA/B TestingStatistical AnalysisMachine Learning

Similar roles

ML Engineering jobs
Pave

Staff AI Engineer

PaveSan Francisco, CA +2

Leads 0→1 AI initiatives from concept to production, building systems with LLMs, agentic architectures, retrieval, and evaluation. Requires 7+ years experience, strong product intuition, and cross-team technical leadership.

221k – 299k/yr
Hybrid7+ YOEML Engineering
Stuut

Member of Technical Staff — Audio and Voice AI

StuutSan Francisco, CA +1

Build and deploy production-grade real-time voice and audio AI systems, including intelligent voice agents, speech-driven workflows, and LLM-powered conversational experiences for financial operations and collections at Stuut. Requires 5+ years software engineering with 2+ years in applied speech/audio AI, experience shipping customer-facing voice systems, and fluency in Python and ML frameworks.

220k – 320k/yr
On-site5+ YOEML Engineering
Perplexity

Member of Technical Staff

PerplexitySan Francisco, CA

Build AI agents that navigate digital environments and perform user tasks. Requires strong AI/ML experience, Python proficiency, and product intuition.

220k – 405k/yr
On-site5+ YOEML Engineering
Perplexity

Member of Technical Staff

PerplexitySan Francisco, CA +1

ML Engineer building and optimizing production recommendation, ranking, and personalization systems that integrate LLMs for Perplexity's AI product.

220k – 405k/yr
On-site5+ YOEML Engineering
Perplexity

Member of Technical Staff

PerplexitySan Francisco, CA

Build and own multimodal AI product and platform systems across the stack at Perplexity. Requires production systems experience, full-stack capability, and strong product judgment.

220k – 405k/yr
On-site5+ YOEML Engineering