Research Engineer, Post-Training
Research engineer focused on post-training LLMs and agents for legal work. Requires hands-on experience training open-weight models and strong Python/research engineering skills.
About the job
What You'll Do
- Drive post-training experiments, pushing agent performance while navigating the Pareto frontier of cost, latency, security, and governance.
- Optimize agent harnesses, including domain-specific skills, tools, subagents, retrieval strategies, and validation loops that improve quality on long-horizon legal work.
- Design and develop grading and reward systems that are reliable enough for evaluation, efficient enough for iteration, and strict enough for high-stakes legal work.
- Study agent behavior, identifying patterns that correlate with successful work product, and converting those findings into training data, evals, or harness changes.
- Work with Harvey researchers and external research partners to define experiments, evaluate methodology, review results, and keep projects moving toward concrete model improvements.
What You Have
- Hands-on experience with post-training or model-training work, such as SFT, preference optimization, RLHF/RLAIF, reward modeling, distillation, or adapting open-weight models to specialized domains.
- Strong judgment about model behavior: you can read traces, inspect outputs, identify failure modes, and reason about whether a metric is measuring the thing that matters.
- Strong Python and research-engineering ability. You can write clean code, debug experiments, and build the simple but reliable systems needed to make research move faster.
- Ability to self-manage ambiguous applied research projects and communicate clearly with researchers, engineers, product teams, domain experts, and external partners.
Nice to Have
- Experience building data or evaluation infrastructure for ML workflows, such as dataset curation pipelines, model-output processing, experiment tracking, evaluation dashboards, or regression analysis tooling.
- Experience with distributed training, inference systems, GPU workloads, or large-scale ML experimentation.
- Research publications, open-source contributions, or shipped industry work in LLMs, agents, evaluation, or ML systems.
Skills
Python, Sft, RLHF, Rlaif, Preference Optimization, Reward Modeling, Distillation, LLMs, Agents, Model Training, Evaluation, Ml Systems
Similar jobs
ML Engineering jobs
Garner Health
New York, NY
Build and deploy algorithmic systems for high-impact healthcare problems, choosing among machine learning, optimization, heuristics, and hybrid approaches. The role requires 4+ years of relevant industry experience, strong applied problem-solving and evaluation skills, and fluency in modern ML tooling.
Cinder
New York, NY
Build and operate production machine-learning systems for content safety, from messy customer data through classification, evaluation, and inference. The role requires 5+ years of ML engineering experience, strong Python and MLOps skills, and sound judgment across classical models and LLMs.
Perplexity
San Francisco, CA
Build AI agent harnesses, models, and product capabilities that enable agents to perform complex work across digital environments. The role combines applied AI research and software engineering, requiring Python proficiency, strong product judgment, and experience with agent tooling, reinforcement learning, or browser technologies.
Mercor
San Francisco, CA
Builds the platform, verifiers, environments, and grading infrastructure used to evaluate enterprise AI agents at scale. The role combines strong software engineering with expertise in agent runtimes, evaluation design, benchmarks, and production failure analysis.
Applied Intuition
Sunnyvale, CA
Optimizes distributed machine learning training and high-throughput offline inference across large accelerator clusters. The role focuses on profiling, scaling efficiency, cluster goodput, GPU performance, and cost-effective processing of autonomy data.