Member of Technical Staff
Build replayable enterprise environments, evaluation systems, graders, and post-training workflows for AI agents. The role spans machine-learning research and production engineering and requires 1–7 years of software or ML systems experience.
About the job
Responsibilities
- Build an environment factory that converts recorded enterprise data and task definitions into runnable environments.
- Design graders that turn ambiguous business objectives into verifiable rewards.
- Develop methods for mining useful tasks, trajectories, and evaluation cases from historical workflows.
- Create representative, reproducible eval sets resistant to overfitting.
- Find combinations of models, tools, context, and policies that maximize performance while reducing inference cost.
- Train and evaluate agents operating over long horizons, incomplete information, and large tool spaces.
- Build replay and observability systems that make agent behavior explainable and measurable.
- Scale from individual environments to thousands of concurrent training and evaluation runs.
- Contribute to research direction and production systems, working directly with the CTO and deploying into enterprise workflows.
Requirements
- 1–7 years of experience building production software or machine-learning systems.
- Experience building strong software and systems that process large, messy datasets at scale.
- Ability to turn ambiguous business objectives into reliably evaluable tasks and signals.
- Understanding of how environment design, reward design, context, tooling, and policy behavior interact.
- Ability to diagnose whether model limitations originate in the model, context, tools, harness, or training.
- Ability to move between research questions and production implementation.
- Commitment to reproducibility, observability, and understanding model behavior.
- Demonstrated work and thoughtful problem-solving are valued over credentials or conventional career paths.
Nice to Have
- Experience with reinforcement-learning environments.
- Experience with LLM post-training.
- Experience with evaluation infrastructure.
- Experience with agent harnesses or closely related systems.
Compensation and Benefits
- Salary: $165,000–$325,000 per year.
- Significant equity and ownership.
- Equinox membership.
- Free meals, coffee, and snacks.
- Health insurance.
- Unlimited paid time off.
Skills
Python, Reinforcement Learning, Llm Post-Training, Evaluation Infrastructure, Agent Harnesses, Machine Learning, Production Software, Large-Scale Data Processing, Observability, Reproducibility
Similar jobs
ML Engineering jobs
Nuro
Mountain View, CA
Build and maintain machine learning infrastructure for autonomy teams, including model pipelines, observability, inference serving, and compiler platforms. The role requires a relevant degree, at least one year of experience, strong Python skills, and familiarity with C++.
Nuro
Mountain View, CA
Build and operate the infrastructure powering large-scale machine-learning training for autonomous-driving systems. The role requires Python proficiency, Kubernetes production experience, distributed-systems expertise, and ownership of reliability, observability, and operational maturity.
Fireworks AI
San Mateo, CABuild, deploy, and optimize AI applications and machine learning models for customer use cases while contributing to an internal ML platform. This new graduate role requires a technical master’s degree, hands-on ML or LLM experience, and strong customer communication skills.
Garner Health
New York, NY
Build and ship production algorithmic systems that improve healthcare quality, access, and cost outcomes. The role combines machine learning, optimization, experimentation, and LLM productionization, requiring at least two years of relevant industry or advanced-degree experience.
SentiLink
United States
Build and deploy production machine-learning models for fraud detection, identity verification, and financial risk products. The role suits new PhD graduates or early-career researchers with strong quantitative foundations, Python experience, and interest in owning the full ML lifecycle.