Member of Technical Staff
Build production infrastructure for replayable enterprise environments, agent evaluation, and continuous model improvement. The role combines hands-on customer deployment, research experimentation, large-scale data processing, and production software engineering.
About the job
Responsibilities
- Forward deploy with customers to understand their tasks and data.
- Develop an environment factory that converts recorded enterprise data and task definitions into runnable environments.
- Design graders that turn ambiguous business objectives into verifiable rewards.
- Mine useful tasks, trajectories, and evaluation cases from historical workflows.
- Experiment with learning objectives and task design.
- Create representative, reproducible eval sets resistant to overfitting.
- Find effective combinations of models, tools, context, and policies while reducing inference cost.
- Build replay and observability systems that make agent behavior explainable and measurable.
- Own production systems, deploy into enterprise workflows, and work directly with the CTO and customers.
Requirements
- Experience building production-grade software; internships are acceptable, or experience in a quant role.
- Strong software development skills and ability to build systems that process large, messy datasets at scale.
- Strong intuition for systems, abstraction, and data structures.
- Ability to turn fuzzy business objectives into reliably evaluable tasks and signals.
- Ability to diagnose whether model limitations arise from the model, context, tools, harness, or training.
- Ability to move between research questions and production implementation.
- Care for reproducibility, observability, and understanding model behavior.
Benefits
- Significant equity and ownership.
- Equinox membership.
- Free meals, coffee, and snacks.
- Health insurance.
- Unlimited PTO.
Skills
Python, Reinforcement Learning, Post-Training, Context Engineering, Agent Engineering, Data Structures, Large-Scale Data Processing, Model Evaluation, Observability, Data Mining
Similar jobs
ML Engineering jobs
Lyft
San Francisco, CADesign, deploy, and improve real-time machine learning systems for Lyft’s ride fulfillment and marketplace products. The role requires 2+ years of ML experience, production programming skills, and expertise with deep learning and recommendation systems.
San Francisco, CA
Develop and deploy responsible AI and machine learning fairness solutions across Pinterest’s large-scale, user-facing products, including generative AI, search, and recommendations. The role requires production ML experience, expertise in fairness interventions and modern architectures, and a master’s or PhD in computer science or a related field.
Benchling
San Francisco, CA
Build datasets, evaluations, and scalable data systems that improve frontier AI models on challenging biological and scientific tasks. The role partners with scientists and AI labs and requires at least two years of experience applying biology and AI, plus hands-on LLM experience.
DataVisor
Mountain View, CA
Builds high-scale data pipelines, distributed systems, and AI agent workflows using LLMs for fraud intelligence platform. Requires 2+ years software engineering, Python proficiency, big data tools, AWS/K8s, and ML foundations.
Garner Health
New York, NY
Build and ship production algorithmic systems that improve healthcare quality, access, and cost outcomes. The role combines machine learning, optimization, experimentation, and LLM productionization, requiring at least two years of relevant industry or advanced-degree experience.