# Research Engineer, LangSmith Engine

**Company:** [LangChain](https://hotfix.jobs/companies/langchain)
**Location:** New York, NY, San Francisco, CA
**Role:** AI Research
**Experience:** 4+ years
**Skills:** Machine Learning, Artificial Intelligence, LLMs, AI Agents, Benchmarking, Model Evaluation, Prompt Engineering, Fine-Tuning, Reinforcement Learning, Preference Optimization, Sft, RLHF, Model Serving, Distributed Systems, Gpu Infrastructure
**Posted:** 2026-08-14

> Research Engineer improving the capability, efficiency, and reliability of autonomous AI agents through benchmarks, experiments, prompting, model optimization, and post-training. The role requires 4+ years of ML/AI research experience, strong software engineering skills, and a master’s or Ph.D. in a relevant scientific field.

## Job Description

## Responsibilities
- Build and maintain benchmarks and evaluations that measure the quality and efficiency of Engine agents on real-world tasks.
- Design and run experiments to improve agent performance across models, prompting, context, tools, orchestration, and agent strategies.
- Explore and implement post-training and fine-tuning techniques when they can meaningfully improve agent capabilities, quality, or cost.
- Turn successful experiments into production improvements, working closely with engineers and researchers to measure impact and prevent regressions.
- Help define the ML roadmap and technical direction for improving Engine agents.
- Mentor other engineers through strong technical leadership.

## Requirements
- 4+ years of experience in ML/AI research or a closely related field.
- Master’s or Ph.D. in a relevant scientific field.
- Hands-on experience working with LLMs and AI agents, including analyzing model behavior and improving real-world performance.
- Strong experience designing benchmarks, evaluations, and experiments for AI/ML systems.
- Strong software engineering skills, with a track record of taking ideas from research prototype to measurable production impact.
- Strong research judgment, ability to work through ambiguity, move quickly, and communicate findings clearly.

## Nice to Have
- Ph.D. in Machine Learning, Computer Science, or Physics.
- Experience with LLM-as-a-judge, automated graders, synthetic data generation, or human evaluation.
- Experience with reinforcement learning, preference optimization, SFT, RLHF/RLAIF, or other post-training techniques for LLMs.
- Experience optimizing LLM agents for cost, latency, or task efficiency in production.
- Experience with model serving, inference optimization, distributed systems, or GPU infrastructure.

## Compensation and Benefits
- Competitive compensation including base salary, variable compensation for relevant roles, meaningful equity, benefits, and perks.
- Medical, dental, and vision coverage.
- Flexible vacation.
- 401(k) plan.
- Life insurance.
- Meals on in-office days in the US.
- Benefits and offerings vary by role, level, and location.

## Similar jobs

- [AI Researcher](https://hotfix.jobs/jobs/077574a5-3c6d-4634-a23c-4a909ce8aa65) - Improbable - Remote
- [Research Engineer, Takeoff Intel](https://hotfix.jobs/jobs/398824a2-65cc-4e28-aaeb-26c3b6610876) - Anthropic - San Francisco, CA - $350k – $850k/yr
- [Applied AI Research Scientist](https://hotfix.jobs/jobs/93baef6f-91a8-4c62-acaa-44c3ea48b467) - Sardine - Remote
- [Researcher, Agent Safety, Oversight and System Mitigations](https://hotfix.jobs/jobs/4544e3bb-bb96-43d2-a96c-cd364b641660) - OpenAI - San Francisco, CA - $380k – $500k/yr
- [Researcher, Agent Safety, Training and Evaluations](https://hotfix.jobs/jobs/d80336da-e453-4999-9f26-85a125b679d9) - OpenAI - San Francisco, CA - $380k – $500k/yr

**Apply:** https://hotfix.jobs/jobs/29548e77-da60-4597-9aff-962a69d238d8
**Canonical:** https://hotfix.jobs/jobs/29548e77-da60-4597-9aff-962a69d238d8