Software Engineer, Applied AI
Build and ship production AI agents and the platform infrastructure that makes them reliable, steerable, and measurable. The role requires strong backend fundamentals, production LLM or agent experience, and expertise in evaluations, retrieval, orchestration, or tool-use design.
About the job
Responsibilities
Agent products
- Design and iterate on agent behavior across real go-to-market workflows, such as sourcing a total addressable market list through search, audience building, and enrichment.
- Map manual, multi-step workflows and turn them into agent-driven flows that match or exceed human performance.
- Build and run evaluations that measure whether agents complete tasks correctly, identify regressions, and uncover failure modes.
- Analyze production failures and systematically improve robustness.
- Partner with product to take agent flows from prototype through beta and general availability, defining quality standards.
Agent platform and infrastructure
- Build the core agent harness, including memory systems, tool infrastructure, and retrieval architecture.
- Improve agent performance through prompting strategies, tool-use design, and context construction.
- Design guardrails and safety checks for predictable production behavior.
- Build a cross-surface evaluation framework for measuring quality, regressions, and performance.
- Create feedback loops that use production logs and real usage to improve prompts, tools, and evaluation coverage.
- Support teams building managed-agent variants for specific use cases.
Requirements
- Experience building or shipping production systems with LLMs or agents, including prompting, tool-use design, agent orchestration, retrieval, structured extraction, or fine-tuning.
- Strong backend fundamentals in APIs, databases, and distributed systems.
- Experience designing model or agent evaluations, measuring regressions, or converting qualitative quality questions into measurable signals.
- Systems-and-outcomes mindset focused on user value and product reliability.
- Comfort debugging real-world failures and shipping iterative improvements quickly.
Nice to have
- Experience with agent frameworks, tool-calling systems, or retrieval architectures such as vector search, hybrid search, and retrieval-augmented generation.
- Experience building or maintaining evaluation or benchmark infrastructure for LLM systems, or running fine-tuning in production.
- Experience with go-to-market, sales, or marketing workflows, including lead sourcing, enrichment, or audience building.
- Familiarity with React, TypeScript, Python, AWS, Aurora/Postgres, ECS/Fargate, Lambda, OpenSearch, ElastiCache/Redis, Terraform, and Datadog.
- Growth mindset and willingness to invest in team learning.
Skills
LLMs, AI Agents, Python, TypeScript, React, APIs, Databases, Distributed Systems, Retrieval-Augmented Generation, Vector Search, AWS, Terraform, Datadog, Redis, Opensearch
Similar jobs
ML Engineering jobs
Clay
San Francisco, CA
Build and ship production machine-learning systems that learn from customer data and behavior, including recommendations, LLM-powered features, evaluation systems, and ML infrastructure. The role requires 5+ years of ML engineering or ML-heavy software engineering experience and strong production systems expertise.
Mirage
New York, NY
Build and deploy agentic systems that power AI-driven creative video workflows. The role requires 5+ years of experience, production ML or agentic pipeline development, context engineering, and expertise in evaluation and agent infrastructure.
Mirage
New York, NY
Build and advance agentic machine-learning systems for multimodal creative tasks, with a focus on video understanding, reasoning, control, and tool use. The role requires strong production ML or agent-pipeline experience and deep knowledge of modern LLM techniques.
Taste Labs
San Francisco, CA
Build evaluation methods, RL environments, agent tooling, and scalable infrastructure that make subjective qualities such as design and taste measurable for frontier AI models. The role requires experience with evaluations, RL environments, ML or post-training, plus strong backend engineering skills.
Mirage
New York, NY
Build and scale generative video and multimodal models, optimizing training and inference for efficiency, throughput, and ultra-low latency. The role requires deep learning systems expertise, strong PyTorch/CUDA experience, and the ability to move research models into production.