Software Engineer, Applied AI
Builds and improves core AI agent systems for retrieval, tool use, document understanding, and orchestration in production. Designs evals, analyzes traces, and iterates based on real enterprise workflows using Python and LLM expertise.
About the job
What You'll Do
- Build and improve the core systems behind our agents across retrieval, tool use, document understanding, memory, and orchestration
- Design evals and experiments that help us understand agent quality in production
- Turn traces, failures, and user behavior into concrete product and architecture decisions
- Work closely with operations, GTM, and deployed teams to understand real workflows and where agents break down
- Evaluate models, prompts, and system designs across real enterprise tasks
- Own the loop from idea -> implementation -> measurement -> iteration
What We're Looking For
- Strong engineering fundamentals and the ability to ship production systems
- Fluency in Python
- Experience building or working on LLM-powered products, agent systems, or adjacent applied AI systems
- An empirical mindset — you reach for logs, traces, experiments, and real usage before guessing
- Strong systems taste — you understand that retrieval, prompting, memory, tools, and UX interact
- High ownership and comfort working in ambiguity
- Strong opinions about what makes agent systems actually work
Strong Candidates May Also Have
- Experience with retrieval, search, or ranking systems
- Experience designing evals, benchmarks, or feedback loops for LLM systems
- Experience building internal tools, workflow products, or operator-facing systems
- Experience in startups or other high-ownership environments
Compensation
- $175,000-$290,000 base salary, plus equity
- Early-stage equity
- Competitive, top-of-market salary
- Catered lunch and dinners
Skills
Python, LLMs, Retrieval, Agents, Prompting, Orchestration, Evals, Document Understanding, Tool Use, Memory Systems
Similar jobs
ML Engineering jobs
Mirage
New York, NY
Build and deploy agentic systems that power AI-driven creative video workflows. The role requires 5+ years of experience, production ML or agentic pipeline development, context engineering, and expertise in evaluation and agent infrastructure.
Mirage
New York, NY
Build and advance agentic machine-learning systems for multimodal creative tasks, with a focus on video understanding, reasoning, control, and tool use. The role requires strong production ML or agent-pipeline experience and deep knowledge of modern LLM techniques.
Taste Labs
San Francisco, CA
Build evaluation methods, RL environments, agent tooling, and scalable infrastructure that make subjective qualities such as design and taste measurable for frontier AI models. The role requires experience with evaluations, RL environments, ML or post-training, plus strong backend engineering skills.
Mirage
New York, NY
Build and scale generative video and multimodal models, optimizing training and inference for efficiency, throughput, and ultra-low latency. The role requires deep learning systems expertise, strong PyTorch/CUDA experience, and the ability to move research models into production.
Clay
New York, NY
Build and ship production AI agents and the platform infrastructure that makes them reliable, steerable, and measurable. The role requires strong backend fundamentals, production LLM or agent experience, and expertise in evaluations, retrieval, orchestration, or tool-use design.