# Software Engineer

**Company:** [Neon Health](https://hotfix.jobs/companies/neon-health)
**Location:** Remote
**Role:** ML Engineering
**Salary:** $140k – $200k/yr
**Experience:** 2+ years
**Skills:** TypeScript, LLMs, AI Agents, Git, Cursor, Github Copilot, Claude, LangChain, Prompt Engineering, Llm Evals, Agent Orchestration, Distributed Systems
**Posted:** 2025-07-21

> Builds and debugs AI agent infrastructure for healthcare automation, including prompt engineering, runtime issue tracing, evaluation datasets, simulation tooling, data pipelines, and observability dashboards. Requires 2-7 years experience with production LLMs/AI agents and TypeScript proficiency.

## Job Description

## What You’ll Do
- Work across our AI agent platform — writing prompts, debugging runtime issues, building agent simulation tooling, creating evals, interfacing with client data, and helping monitor system behavior at scale.
- Trace and fix runtime bugs, then write regression tests.
- Design evaluation datasets to simulate realistic workflows or red-team our system.
- Build internal tooling for QA and agent simulation.
- Normalize and transform messy client data for system integration.
- Set up automatic testing and latency tracking infrastructure.
- Create dashboards and observability tooling for agentic system behavior.
- Expand on our existing eval & testing framework and agent simulation infrastructure.

## Skills Required

**Technical Skills**
- Proficiency in **TypeScript**
- Strong generalist software engineering
- Strong debugging skills (trace runtime failures, dig through logs, pinpoint issues in async or multi-step agent systems)
- Data transformation and ingestion (build pipelines to normalize and convert unstructured data for AI systems)
- Strong understanding of system design, including distributed systems and reliability/performance tradeoffs
- Experience using modern AI coding tools (e.g. **Cursor**, **GitHub Copilot**, **Claude**)
- Excellent documentation and testing discipline
- Proficiency with **Git**

**Soft Skills**
- Care about improving agent behavior
- High agency; thrive with minimal structure
- Comfortable getting in the weeds with details, edge cases, editing prompts, writing evals
- Comfortable with ambiguity; work well with loose specs spanning prompts, code, RLHF
- Learn fast and move fast; pattern-match from past systems work to LLM edge cases

## Experience & Who Should Apply
- 2-7 years of experience working closely with **LLMs** or **AI agents** in production systems
- Created internal tools or frameworks for QA, evals, or agent simulation
- Contributed to fast-paced product cycles involving AI behavior, latency, user experience

## Nice to Have
- Experience with multi-agent systems, TTS/NLP pipelines, or structured output validation
- Familiarity with testing frameworks, **LangChain**-style agent orchestration, or in-house eval harnesses
- Experience with **prompt engineering**, **LLM evals**, and agent orchestration

## Similar roles

- [Software Engineer](https://hotfix.jobs/jobs/6d8b2f8e-891e-4d59-b0b0-632d28fa07ac) - Traba - New York, NY - $140k – $200k/yr
- [Forward Deployed Engineer, RL Environments](https://hotfix.jobs/jobs/eff0518b-0791-4ec5-8063-9b548d70aa46) - Labelbox - San Francisco, CA - $140k – $200k/yr
- [Machine Learning Engineer, Search Quality](https://hotfix.jobs/jobs/0165787d-efc5-4fa3-9f98-5fcfdd542c85) - Glean - $140k – $265k/yr
- [Machine Learning Engineer II, Computer Vision Applied Science](https://hotfix.jobs/jobs/6084e23c-ceea-4ad5-8d59-88bf08627c8a) - Pinterest - Remote - $139k – $286k/yr
- [Machine Learning Engineer, Core Engineering](https://hotfix.jobs/jobs/296145d5-cbc5-4031-8ff2-03c1c5d52c27) - Pinterest - Remote - $139k – $286k/yr

**Apply:** https://hotfix.jobs/jobs/66a4542e-7f62-40d3-ae76-bf096ecd50bd
**Canonical:** https://hotfix.jobs/jobs/66a4542e-7f62-40d3-ae76-bf096ecd50bd