Research Engineer, Post-Training
Research Engineers at Distyl build and productionize post-training techniques (fine-tuning, RLHF, reward models, evals) to improve reliability and behavior of compound AI systems for enterprise customers. Requires strong applied ML experimentation skills and ownership of real-world outcomes.
About the job
Key Responsibilities
- Design and run post-training workflows that improve the behavior, reliability, and usefulness of AI systems
- Develop datasets, preference signals, evaluation suites, reward models, fine-tuning workflows, and feedback loops for applied AI use cases
- Investigate how different post-training techniques affect system behavior across enterprise workflows and production constraints
- Build infrastructure for experimentation, model comparison, regression testing, and behavior analysis
- Partner with AI Researchers to explore new post-training methods and with AI Engineers to apply successful techniques in deployed systems
- Analyze model outputs, failure modes, human feedback, and production traces to identify opportunities for behavioral improvement
- Create repeatable processes for adapting AI systems to customer domains while preserving robustness, transparency, and maintainability
- Communicate clearly with internal teams and customer stakeholders about model behavior, evaluation results, limitations, and tradeoffs
Requirements
- Experience improving model behavior through fine-tuning, preference optimization, reinforcement learning, reward modeling, synthetic data, evals, or related post-training techniques
- Strong programming and experimentation skills to build training and evaluation pipelines, run controlled experiments, analyze results, and iterate quickly
- Research-oriented builder mindset focused on understanding why behavior changes
- AI systems mindset understanding that model behavior is shaped by data, prompts, tools, retrieval, evaluators, and deployment context
- AI-native working style using AI tools daily to accelerate coding, analysis, debugging, experimentation, and research
- Bias towards measurement through evaluations, comparisons, regression tests, and production-relevant metrics
- Comfort with applied constraints around cost, latency, reliability, data availability, and customer requirements
- Ownership mentality for whether post-training work improves real system outcomes
Nice-to-Haves
- Experience with compound AI systems in production environments
Compensation
- Base salary range: $150000 – $250000, depending on experience, location, and level
- Eligible for meaningful equity
- Comprehensive benefits package: 100% covered medical, dental, and vision for employees and dependents; 401(k) with additional perks (e.g., commuter benefits, in-office lunch)
Skills
Post-Training, Fine-Tuning, Preference Optimization, Reinforcement Learning, Reward Modeling, Synthetic Data, Evaluation Frameworks, Experimentation, Python, Machine Learning
Similar jobs
ML Engineering jobs
LangChain
New York, NYBuild and teach reliable AI agent systems through customer workshops, technical content, guidance, and reference implementations. The role requires strong Python and agent-development experience plus a background delivering customer-facing technical training.
Roboflow
San Francisco, CA
Build reinforcement-learning environments, evaluations, datasets, and scalable infrastructure for frontier AI capabilities. The role suits a high-agency generalist engineer with experience in agents, evaluations, or RL workflows and strong communication skills.
Beacon Biosignals
Boston, MADevelop and productionize machine- and deep-learning algorithms for biosignal and EEG data used in medical devices, clinical development, and diagnostics. The role requires 4+ years of industry experience, DSP and statistics expertise, PyTorch proficiency, and familiarity with regulated environments and production ML practices.
Applied Intuition
Sunnyvale, CA
Develop and deploy ML-first behavior prediction and planning systems for autonomous vehicles, forecasting the motion and interactions of road users. Requires a bachelor's degree, deep learning lifecycle expertise, and at least three years of production software experience with C++ or Python.
Fab2
Austin, TXBuild the AI platform behind fab2, including model infrastructure, agent systems, evaluations, and tools for engineering and fab operations. The role requires strong production software engineering skills and comfort working across frontend, backend, infrastructure, and data.