Member Of Technical Staff - RL Environments
Build and evaluate reinforcement learning environments and AI agents for enterprise workflows. The role focuses on agent training, capability measurement, verifier and reward design, synthetic data pipelines, and improving agent performance through systematic evaluation.
About the job
Responsibilities
- Build new reinforcement learning (RL) environments targeting different agent capabilities and industry areas.
- Train and evaluate agents in those environments.
- Integrate tasks, data, tool implementations, and verifiers.
- Work across modeling and product to identify gaps in agent performance and improve agents and environments.
- Work with external vendors to create expert-built RL environments and tools that ensure task, data, and verifier quality.
- Automate discovery of model capability gaps and systematically measure agent performance during evaluations and training.
Requirements
- Experience engineering agents and optimizing them for specific industry use cases.
- Experience reviewing agent trajectories to identify failure points and resolving them through model training or harness engineering.
- Strong focus on measuring agent capabilities and turning measurements into repeatable processes.
- Experience defining desirable agent outcomes, implementing verifiers, and tuning reward designs.
- Experience designing and running annotation workflows to analyze agent performance and verify data quality.
- Experience building synthetic data pipelines to scale evaluation and training efforts.
- Regular use of AI agents and experience improving agent workflows and productivity.
Nice-to-haves
- Experience training with reinforcement learning, including scaling, troubleshooting, and tuning environments.
Compensation and Benefits
- Weekly lunch stipend of $75/£75 or equivalent in local currency.
- Full health and dental benefits, including a separate mental health budget.
- RRSP matching, 401(k), and pension scheme.
- Up to six months of 100% parental leave top-up for either parent.
- Annual enrichment benefits for arts and culture, fitness and wellness, quality time, and workspace improvements.
- Education and learning stipend for conferences, courses, and coaching.
- Six weeks of paid vacation (30 working days).
- Travel budget for remote employees to visit other offices and an annual company offsite.
- Coworking benefit for employees not near an office.
- $500 home office stipend.
Skills
Reinforcement Learning, AI Agents, Synthetic Data, Model Training, Agent Evaluation, Annotation Workflows, Data Quality, Reward Design, Verifiers, Python
Similar jobs
ML Engineering jobs
Rollstack
United StatesBuild production AI capabilities for automated slide and document generation, working across LLM applications, data analysis, and content generation. The role requires 3+ years in machine learning and NLP, advanced Python, and experience with LLM frameworks and production systems.
OpenAI
London, United Kingdom
Build and deploy AI-powered products for digital-native customers, taking systems from experimentation through production and scale. The role requires strong Python skills, hands-on production engineering, systematic AI evaluation, and the ability to navigate reliability, security, governance, and customer impact.
Elliptic
London, United Kingdom
Build full-stack AI agent fleets, APIs, workflows, and internal services that automate complex business processes. The role requires at least five years of engineering experience, hands-on LLM framework experience, production AWS expertise, Kubernetes, and strong API and database skills.
Protege
Remote
Build the technical foundation for a new business vertical, creating reusable infrastructure and leading early customer engagements from scoping through delivery. The role requires 3+ years of engineering experience, strong Python and SQL skills, backend/data expertise, and comfort operating in ambiguity.
Build
New York, NYBuild and operate Dougie, an agentic AI system that executes workflows, evaluates its own performance, retains institutional context, and improves in production. The role requires experience deploying unattended agentic systems and engineering reliable memory, retrieval, orchestration, and feedback loops.