Engineering Manager, AI
Leads the engineering team responsible for production agentic infrastructure and evaluation systems, combining people management with hands-on technical direction. The role requires 5+ years of software engineering experience, engineering leadership, and experience shipping AI systems.
About the job
Responsibilities
- Lead and grow an engineering team building and hardening agentic infrastructure.
- Establish engineering culture through clear expectations, 1:1s, feedback, and career development.
- Own the architecture of workflow agents, including orchestration, model serving, latency, cost per task, and failure handling.
- Define reliability standards for autonomous permit submission and build release-gating systems against those standards.
- Build evaluation infrastructure, including golden datasets, LLM-as-judge pipelines, human annotation tools, and online monitoring for drift and regressions.
- Develop internal tooling for testing, error analysis, and iterative improvement of agentic systems.
- Improve reliability, observability, and engineering efficiency across the AI stack.
- Hire and develop engineers with ML/AI systems expertise and future engineering leaders.
Requirements
- 5+ years of software engineering experience, including 1+ year in engineering leadership.
- Direct experience building or shipping a production AI or agentic system, including its supporting infrastructure.
- Hands-on familiarity with evaluation infrastructure such as golden datasets, LLM-as-judge systems, human annotation, or online monitoring.
- Strong system design and architecture fundamentals, including judgment about when to use agents versus simpler systems.
- Ability to lead through ambiguity and turn unclear problems into structured execution.
- Strong technical fluency in code reviews, architecture reviews, and model/system tradeoffs.
- Strong communication skills with technical and non-technical stakeholders.
- High standards for engineering quality, reliability, and performance.
Nice-to-haves
- Experience with workflow-heavy, enterprise SaaS, or infrastructure products.
- Exposure to complex regulated environments such as construction, logistics, or government systems.
Compensation and Benefits
- $180,000–$250,000 annual salary.
- Meaningful equity.
- Company-paid medical, dental, and vision coverage, plus healthcare FSA.
- 401(k) savings plan.
- Unlimited paid time off and paid family leave.
- Home office and equipment stipend.
- Daily in-office lunch and dinner.
- Commuter benefits for transit and parking.
Skills
Ai Systems, Agentic Systems, System Design, Model Serving, Llm Evaluation, Llm-As-Judge, Human Annotation, Online Monitoring, Observability, Workflow Orchestration, Architecture, Machine Learning
Similar jobs
Engineering Management jobs
Permitflow
New York, NY
Leads and grows an engineering team building new products from initial market hypothesis through commercial launch. The role requires at least five years of software engineering experience, engineering leadership, strong architecture judgment, and comfort operating in ambiguous 0-to-1 environments.
Permitflow
New York, NY
Leads and scales an engineering team while remaining hands-on with architecture, system design, code review, and technical execution. The role requires 5+ years of software engineering experience, engineering leadership experience, and strong communication in ambiguous, high-growth environments.
Duolingo
New York, NY
Engineering manager leading a cross-functional product team responsible for strategy, experimentation, reliable infrastructure, and measurable user impact. Requires experience hiring and developing software engineers, delivering multi-engineer projects, and partnering across product, design, and data.
NexHealth
Seattle, WALeads engineering team building scalable DataLake platform for healthcare data ingestion, storage, and secure access at global scale. Requires 4+ years managing teams, 10+ years software engineering, and experience with large-scale infrastructure.
Applied Intuition
Ann Arbor, MI
Leads and mentors a software engineering team building reliable, high-performance simulation products, while guiding technical execution, prioritization, and hiring. Requires a bachelor’s degree, programming experience in C++, Python, or Java, people-management experience, and eligibility for a US Government security clearance.