Software Engineer - AI Automation
Build and operate LLM-powered agents that automate business workflows, integrating systems of record through MCP and validating behavior with structured evaluations. Requires 5+ years of software development experience, strong Python and SQL, and hands-on experience shipping agents to production.
About the job
Responsibilities
- Author reusable skills that encode team processes, judgment, and edge cases for agent invocation.
- Build agent workflows by connecting agents to systems of record through MCP servers, composing skills and tools, and extending MCP coverage when needed.
- Define correctness criteria with business stakeholders, create evaluation sets, establish baselines, and use evaluations to guide release decisions.
- Deploy and support agents in production, including permissions, human-in-the-loop checkpoints, audit trails, rollback mechanisms, tuning, documentation, and business-team ownership.
- Partner with data engineers on data access, models, and data quality requirements.
Requirements
- Substantial hands-on experience shipping LLM-backed applications involving prompting, tool use, agentic loops, and multi-step workflows.
- At least one shipped agent used by non-engineers.
- Practical experience with MCP, skill or tool authoring, and orchestrating agents against systems of record.
- Experience building evaluation sets and using them to make product decisions.
- Fluency with agentic coding tools such as Cursor.
- Strong Python skills and comfort with SQL and production data models.
- Ability to translate business processes into technical requirements without a product manager.
- Comfort working with ambiguity, undocumented processes, internal tools, and unclear ownership.
- Experience with Git, CI, containers, deployment, and operational support.
- At least 5 years of hands-on software development experience.
Nice-to-haves
- Experience building or maintaining MCP servers.
- Solutions engineering, professional services, internal tools, or startup founding experience.
- Familiarity with Linear, Jira, Zendesk, Salesforce, or Snowflake.
- Experience in insurance, fintech, or another regulated domain.
Compensation
- US base salary: $115,000-$168,000 per year, depending on geographic market and qualifications.
Benefits
- 100% medical, dental, and vision coverage.
- Flexible paid time off.
- Annual home office stipend and WeWork access.
- Mental and physical health wellness programs.
Skills
Python, SQL, Mcp, Agentic Workflows, Llm Applications, Prompting, Tool Use, Evaluation Sets, Cursor, Git, Continuous Integration, Containers, Linear, Jira, Zendesk
Similar jobs
ML Engineering jobs
Build
New York, NY
Build modular AI operations and evaluation systems that power complex real estate workflows. The role focuses on improving output quality, defining correctness with domain experts, and reducing human review while maintaining high standards.
Build
New York, NYBuild and operate Dougie, an agentic AI system that executes workflows, evaluates its own performance, retains institutional context, and improves in production. The role requires experience deploying unattended agentic systems and engineering reliable memory, retrieval, orchestration, and feedback loops.
Build
New York, NYBuild and ship production agentic AI workflows for complex real estate and built-world processes. The role combines product engineering, applied AI, customer collaboration, workflow orchestration, evaluation, and reliable user-facing experiences.
Mintlify
San Francisco, CA
Build and own customer-facing AI products from experimentation through production, including reliable agents, evaluation systems, APIs, interfaces, and infrastructure. Requires at least four years of software development experience and deep production experience with language-model systems.
PathAI
Boston, MADevelop and deploy machine learning models for biomedical research and AI products, collaborating with scientific, engineering, and product teams. Requires an advanced quantitative degree, substantial ML experience, Python proficiency, and experience bringing models into production or research applications.