# Staff Applied AI Scientist

**Company:** [Order.co](https://hotfix.jobs/companies/orderco)
**Location:** Remote
**Role:** AI Research
**Experience:** 10+ years
**Skills:** Machine Learning, LLMs, AI Agents, Machine Learning Operations, Model Serving, Retrieval, Embeddings, Vector Search, Prompt Engineering, Model Evaluation, CI/CD, Drift Detection, AWS, GCP, Statistical Reasoning
**Posted:** 2026-08-26

> Own the architecture, delivery, evaluation, and production operations of AI capabilities embedded in procurement and finance workflows. The role requires 10+ years in applied AI or machine learning, deep LLM and agent expertise, and experience delivering measurable production outcomes.

## Job Description

## Responsibilities
- Own end-to-end AI and machine learning architecture, including model hosting and serving, prompt and model versioning, retrieval and embeddings, agent tooling, guardrails, and evaluation.
- Choose deterministic, large language model, or agent-based approaches based on accuracy, latency, cost, and reliability trade-offs.
- Build evaluation systems connecting offline and online quality to business outcomes and risk controls.
- Own experimentation, versioning, CI/CD for models and prompts, monitoring, drift detection, rollback, rollout strategy, and incident readiness.
- Implement safety guardrails, hallucination mitigation, bias testing, and sensitive-data handling.
- Define AI-ready data requirements, including training and retrieval data, labeling, feature availability, and vector or search infrastructure; partner with data engineering and platform teams to implement them.
- Prioritize AI opportunities, define hypotheses and success criteria, and sequence execution across a portfolio.
- Advise product and engineering leadership on feasibility, cost, risk, and expected return; establish reusable architecture and delivery patterns.
- Mentor experienced individual contributors on applied AI execution and production quality.
- Deliver predictive ordering, agentic workflow copilots, and evaluation and operations capabilities for procurement and finance workflows.

## Requirements
- 10+ years of experience in applied data science, machine learning, or applied AI.
- Repeated delivery of production systems that measurably improved a business metric.
- Ownership of AI and machine learning system architecture, including serving, retrieval, evaluation, guardrails, and operational processes.
- Deep knowledge of current large language model and agent technologies, their failure modes, evaluation methods, and appropriate use cases.
- Experience with machine learning operations, including versioning, CI/CD for models and prompts, monitoring, drift detection, and rollback.
- Portfolio-level prioritization of competing AI opportunities and development of execution plans.
- At least 18 months of daily use of AI-native engineering workflows across design, coding, debugging, and review.
- Experience establishing model governance, monitoring, and responsible AI standards.
- Working implementation proficiency across at least two cloud or technical ecosystems, such as AWS and Google Cloud.
- Strong quantitative foundation in experimentation, statistical reasoning, and causal thinking.
- Ability to align product, engineering, and operations stakeholders on sequencing and trade-offs in ambiguous situations.

## Preferred Qualifications
- Retrieval systems, vector search, ranking, recommendation, or production personalization experience.
- Self-hosted or local AI infrastructure, including self-managed agent environments.
- Experience in e-commerce, B2B procurement, vendor management, financial products, or integrations with external systems.

## Success Measures
- Multiple AI capabilities are live in production with clear hypotheses and measured outcomes within the first 6–9 months.
- Established model architecture and evaluation approaches are adopted by other initiatives.
- Monitoring, drift detection, rollback, and incident playbooks operate reliably under real conditions.

## Similar jobs

- [Senior Staff Software Engineer, Autonomy Capabilities](https://hotfix.jobs/jobs/178a5273-2769-4ad0-94d2-e6b78ca41ab8) - Shield AI - San Mateo, CA - $281k – $421k/yr
- [Senior Staff Engineer, Autonomy Capabilities – Maritime](https://hotfix.jobs/jobs/f17e0e39-2642-4316-9ba2-6844b72e7c29) - Shield AI - Washington, DC - $221k – $331k/yr
- [Staff Machine Learning Model Risk Specialist](https://hotfix.jobs/jobs/feed0bc4-cd1d-4345-a84c-af5e2e91bcb8) - Upstart - Remote - $140k – $218k/yr
- [Staff+ Researcher, Cybersecurity Products](https://hotfix.jobs/jobs/f9a7748f-2310-4a86-a81d-d17949dd1879) - Anthropic - San Francisco, CA - $405k – $485k/yr
- [Senior Research Engineer, Safety](https://hotfix.jobs/jobs/895361c1-4855-4f5a-8be7-da7e4af996f6) - Decagon - San Francisco, CA - $200k – $400k/yr

**Apply:** https://hotfix.jobs/jobs/a188f352-29c8-4ce3-b524-b0fcdd4d4fc6
**Canonical:** https://hotfix.jobs/jobs/a188f352-29c8-4ce3-b524-b0fcdd4d4fc6