Skip to content
Neon HealthNeon HealthUnited States

Software Engineer

Builds and debugs AI agent infrastructure for healthcare automation, including prompt engineering, runtime issue tracing, evaluation datasets, simulation tooling, data pipelines, and observability dashboards. Requires 2-7 years experience with production LLMs/AI agents and TypeScript proficiency.

140k – 200k
Remote2+ YOEML Engineering

About the role

What You’ll Do

  • Work across our AI agent platform — writing prompts, debugging runtime issues, building agent simulation tooling, creating evals, interfacing with client data, and helping monitor system behavior at scale.
  • Trace and fix runtime bugs, then write regression tests.
  • Design evaluation datasets to simulate realistic workflows or red-team our system.
  • Build internal tooling for QA and agent simulation.
  • Normalize and transform messy client data for system integration.
  • Set up automatic testing and latency tracking infrastructure.
  • Create dashboards and observability tooling for agentic system behavior.
  • Expand on our existing eval & testing framework and agent simulation infrastructure.

Skills Required

Technical Skills

  • Proficiency in TypeScript
  • Strong generalist software engineering
  • Strong debugging skills (trace runtime failures, dig through logs, pinpoint issues in async or multi-step agent systems)
  • Data transformation and ingestion (build pipelines to normalize and convert unstructured data for AI systems)
  • Strong understanding of system design, including distributed systems and reliability/performance tradeoffs
  • Experience using modern AI coding tools (e.g. Cursor, GitHub Copilot, Claude)
  • Excellent documentation and testing discipline
  • Proficiency with Git

Soft Skills

  • Care about improving agent behavior
  • High agency; thrive with minimal structure
  • Comfortable getting in the weeds with details, edge cases, editing prompts, writing evals
  • Comfortable with ambiguity; work well with loose specs spanning prompts, code, RLHF
  • Learn fast and move fast; pattern-match from past systems work to LLM edge cases

Experience & Who Should Apply

  • 2-7 years of experience working closely with LLMs or AI agents in production systems
  • Created internal tools or frameworks for QA, evals, or agent simulation
  • Contributed to fast-paced product cycles involving AI behavior, latency, user experience

Nice to Have

  • Experience with multi-agent systems, TTS/NLP pipelines, or structured output validation
  • Familiarity with testing frameworks, LangChain-style agent orchestration, or in-house eval harnesses
  • Experience with prompt engineering, LLM evals, and agent orchestration

Skills

TypeScriptLLMsAI AgentsGitCursorGithub CopilotClaudeLangChainPrompt EngineeringLlm EvalsAgent OrchestrationDistributed Systems

Similar roles

ML Engineering jobs
Traba

Software Engineer

TrabaNew York, NY

Build and own production AI agent systems (harnesses, evals, orchestration) on frontier LLMs for industrial supply chain workflows at Traba. Requires 1+ years shipping LLM/agent features to production, strong Python/TS skills, and high-agency in ambiguous customer environments.

140k – 200k
On-site1+ YOEML Engineering
Labelbox

Forward Deployed Engineer, RL Environments

LabelboxSan Francisco, CA

Builds and maintains sandboxed, reproducible RL environments for AI agent training, including terminal, browser, and tool-augmented setups. Requires 2+ years Python/systems engineering, containerization, and RL concepts understanding.

140k – 200k
Hybrid2+ YOEML Engineering
Glean

Machine Learning Engineer, Search Quality

GleanUnited States

Build advanced search quality systems using machine learning, including personalization signals, ranking models, and domain-adapted LLMs for enterprise search. Requires 2+ years experience in ML, search/NLP, strong coding in Python/Go/Java/C++, and bachelor's in CS/math.

140k – 265k
Hybrid2+ YOEML Engineering
Pinterest

Machine Learning Engineer II, Computer Vision Applied Science

PinterestSan Francisco, CA

Build and fine-tune vision-centric VLMs and generative models using Pinterest's visual-text datasets. Requires 2+ years industry computer vision experience and an M.S. or Ph.D.

139k – 286k
Remote2+ YOEML Engineering
Pinterest

Machine Learning Engineer, Core Engineering

PinterestSan Francisco, CA +2

Build and improve machine learning models for Pinterest's recommendation systems across Homefeed, Ads, Search, and more. Requires 2+ years experience in ML methods like personalization and recommender systems, plus hands-on work with large-scale data pipelines.

139k – 286k
Remote2+ YOEML Engineering