Member of Technical Staff
Build and own frontier AI agent capabilities from model evaluation and rapid prototyping through production launch, monitoring, and iteration. The role requires strong software engineering experience, applied AI/ML product expertise, and familiarity with agent systems, tool use, and long-running task execution.
About the job
Responsibilities
- Evaluate frontier models against real user tasks; identify useful behaviors and failure modes; and turn promising advances into production agent systems.
- Own the lifecycle from rapid prototyping and evaluation through launch, monitoring, and iteration.
- Improve agents’ planning, tool use, context management, error recovery, and long-running task execution.
- Apply state-of-the-art machine learning and large language model techniques to build scalable agent capabilities, including skills, plugins, artifact generation, tool use, auto-research, and multi-agent collaboration.
- Own agent behavior and capabilities end to end across user-facing interfaces and backend services.
- Define offline and online evaluations for task completion, correctness, safety, latency, cost, and user satisfaction.
- Build secure, observable, and reliable agent systems with permissions and safeguards for sensitive actions.
- Develop tracing, replay, and monitoring infrastructure to make agent failures reproducible and actionable.
- Collaborate with product management, data science, and research teams to identify opportunities and productize emerging model capabilities.
- Apply advances in models, inference, evaluation, and agent architecture to improve production performance.
- Set technical direction, participate in design reviews, mentor colleagues, and provide technical leadership.
Requirements
- Typically 6+ years of professional software engineering experience, with a record of building and owning robust AI-powered, large-scale, user-facing, or data-intensive products.
- Strong software engineering fundamentals and experience building and operating AI/ML products, backend services, or distributed systems at scale.
- Experience owning the AI product lifecycle, including data analysis, rigorous evaluation, production monitoring, and iterative improvement.
- Ability to define metrics and use production data and user feedback to guide decisions.
- Practical experience with one or more of agent harnesses, tool use, context engineering, model evaluation, browser automation, or long-running task execution.
- Strong product judgment and execution, including translating ambiguous user needs into applied AI or ML problems and shipping durable solutions with measurable impact.
- Genuine interest in frontier AI capabilities, agent systems, and productizing new model behaviors.
Nice to Have
- Experience with LLM context engineering or harness engineering, subagents, coding assistants, or long-running and autonomous task execution.
- Deep familiarity with current model families across reasoning, tool use, context management, and long-horizon tasks.
- Experience building agent permissions, safeguards, evaluation infrastructure, or production observability systems.
- Experience with mid-training, post-training, or reinforcement learning for frontier or open-source models.
- AI/ML research experience demonstrated through publications, open-source contributions, or other meaningful research impact.
- Experience at a fast-growing startup or on a high-ownership engineering team.
Compensation
- Annual salary range: $220,000–$405,000.
Skills
Python, Go, Rust, Postgres, DynamoDB, AWS, TypeScript, Agent Systems, LLMs, Model Evaluation, Distributed Systems, Reinforcement Learning, Browser Automation, Observability
Similar jobs
ML Engineering jobs
Discord
United States
Build and deploy scalable machine learning systems that detect risk, anomalous behavior, and account-integrity issues to help keep Discord users safe. The role requires 4+ years of ML experience, strong Python and framework expertise, and a relevant bachelor’s or master’s degree.
Fetch
United States
Build and operate low-latency machine learning systems for ad ranking, relevance, and optimization, including feature pipelines, experimentation, evaluation, and production inference. The role requires 6+ years of software engineering experience, strong Python skills, AWS experience, and practical LLM application experience.
Plaid
New York, NYLeads applied research for next-generation fraud detection models across graph, sequential, image, and video data, translating prototypes into production solutions. Requires strong Python skills, research leadership, and a PhD or equivalent research experience.
Plaid
New York, NYSenior Machine Learning Engineer building and scaling production fraud-detection systems, including feature pipelines, online inference, monitoring, and reliability capabilities. Requires 6+ years of experience with ML infrastructure and technologies such as Python, PyTorch, Spark, SageMaker, and Airflow.
Plaid
San Francisco, CADevelop and deploy machine learning models that detect fraud, from feature engineering and experimentation through production optimization. The role requires 7+ years of ML or software engineering experience, strong Python and SQL skills, and expertise in evaluating and scaling reliable models.