Build benchmarks, datasets, and evaluation systems to measure and improve AI model quality for fraud, identity, and risk judgment tasks. Collaborate across research, engineering, and product to drive rigorous experimentation and iteration in high-stakes environments.
250k – 400k/yr
On-siteML Engineering
About the role
Responsibilities
Build proprietary benchmarks and datasets to evaluate models and model systems on fraud, identity, and risk workflows
Design and run offline and online evals that measure model performance on real customer tasks
Define quality metrics for judgment systems, including precision, calibration, consistency, abstention, and failure handling
Study where models and agents break, and turn those failures into better evals, datasets, and training loops
Build reusable evaluation tools and quality building blocks
Partner closely with research, engineering, product, and design to improve system quality
Help create a strong culture of scientific experimentation, clear measurement, and continuous iteration
Requirements
Care deeply about craftsmanship and have strong opinions about model quality, measurement, and experimental rigor
Want to work on core model and agent behavior
Excited by defining what “good” looks like in messy, high-stakes environments
Designs and runs experiments to improve oversight of increasingly capable AI models, including model training, evaluation, and deployment of practical systems. Analyzes failures and develops techniques to train more aligned models using oversight signals.
250k – 445k/yr
HybridML Engineering
Research Scientist / Engineer — Multimodal Agent
Luma AIPalo Alto, CA
Builds and trains large-scale multimodal agentic models involving reasoning, planning, coding, and tool calling. Requires strong ML foundations, PyTorch expertise, and experience with distributed training on massive datasets.
250k – 450k/yr
HybridML Engineering
Research Engineer, Judgment Systems
VarianceSan Francisco, CA
Research Engineer designs evaluations, studies model failures, and builds research loops to improve AI agents for high-stakes fraud detection and judgment tasks. Requires ML training experience, experimental rigor, and strong engineering skills in adversarial environments.
250k – 400k/yr
On-siteML Engineering
AI Engineer
MomentNew York, NY
Builds and deploys AI primitives and agents to automate workflows and enhance user experiences in investment management platform. Requires AI agent experience, distributed systems knowledge, and product-minded engineering across tech stacks.
250k – 325k/yr
On-siteML Engineering
AI Inference Engineer - Model Optimization & Deployment
ZooxFoster City, CA +2
Optimizes and deploys large-scale AI models (LLMs, VLMs) for real-time inference on power-constrained vehicle hardware. Requires expertise in quantization, TensorRT compilation, custom CUDA kernels, and production C++/Python for edge devices.