Senior Full Stack AI Engineer (Rapid Prototyping & Analytics )
Builds and deploys end-to-end AI/ML systems, including LLM workflows and rapid prototypes for internal tools and product features in healthcare. Requires 4+ years experience, strong full-stack skills, evaluation/monitoring expertise, and Python proficiency.
About the job
Responsibilities
- Drive Prompt's mission to improve healthcare through modern technology including AI
- Lead AI projects from ideation → architecture → production → iteration until tools are widely adopted and loved
- Design, build, and deploy end-to-end AI systems across both traditional ML and LLM-based workflows
- Partner with stakeholders across the company to understand workflows, define success criteria, and deliver practical solutions
- Support internal as well as product-facing AI initiatives as needed
- Design and implement evaluation, monitoring, and observability metrics (accuracy, cost, latency, and business impact)
- Build guardrails around LLM pipelines including validation and human-in-the-loop workflows
- Develop rapid proofs-of-concept and prototypes, then harden them into reliable systems
- Help grow and lead the AI team, including training and development of junior team members
Qualifications
- 4+ years of experience building AI/ML systems in industry or academia, with meaningful production ownership
- Strong experience building and operating AI systems end-to-end, not just models in isolation
- Strong experience with LLM-based systems (programmatic prompt optimization, evaluation, prompting, structured outputs, RAG, multi-step workflows)
- Solid understanding of evaluation and measurement for AI systems (offline metrics, online monitoring, regressions, cost tracking)
- Ability to work closely with non-technical stakeholders to translate workflows into effective AI solutions
- Strong software engineering fundamentals and ability to ship maintainable, production-grade systems
- Track record of benchmarking and iteratively improving accuracy, cost, and latency
- Excellent communication skills, with the ability to explain complex AI concepts to non-technical stakeholders
- Python (>3.9)
- Docker (Basics)
- AWS familiarity (S3, RDS, EC2) preferred
- Strong problem-solving mindset and ability to operate effectively in ambiguous, fast-paced settings
- Willingness to learn and deeply understand the healthcare sector, specifically physical therapy, occupational therapy, and speech therapy
Nice-to-Haves
- Experience building internal AI tools or platforms used across multiple business functions
- Experience designing evaluation harnesses for LLM workflows (golden datasets, regression tests, human review loops)
- Experience setting up analytics and dashboards to track AI usage, outcomes, and ROI
- Familiarity with workflow automation and business systems (CRMs, support tools, analytics platforms)
- Experience with FastAPI, Streamlit, Pydantic, SQLAlchemy, PostgreSQL
- Experience with retrieval systems (vector search, hybrid search, grounding and citation strategies)
- Experience building healthcare software and/or data processing pipelines
Skills
Python, LLMs, RAG, Docker, AWS, FastAPI, Streamlit, Pydantic, Sqlalchemy, Postgres
Similar jobs
ML Engineering jobs
LangChain
New York, NY
Leads hands-on development and deployment of production AI agents and workflows, sets technical standards, and mentors engineers. Requires 4+ years of software engineering experience, production LLM application experience, and strong Python or TypeScript skills.
LegitScript
United States
Owns the full lifecycle of data and ML solutions, from ingestion and feature-ready datasets through production deployment and business-impact measurement. The role combines data engineering, applied machine learning, MLOps, and generative AI to build risk detection capabilities.
Mercury
San Francisco, CABuild and operate the platform that deploys, serves, observes, and retrains production machine-learning models for real-time fraud and financial-crime risk decisions. Requires 5+ years of ML engineering, backend, or MLOps experience, strong Python skills, and production model-serving expertise.
Parallel Systems
Los Angeles, CA
Develop and deploy real-time perception and sensor-fusion software for autonomous battery-electric rail vehicles. The role requires strong robotics, geometry-based computer vision, C/C++ and Rust experience, plus hands-on work with multimodal sensors and production systems.
Lightning AI
New York, NYBuild and optimize large language model training and post-training pipelines, improving model quality, distributed performance, evaluation, and production readiness. The role requires deep PyTorch and transformer experience, strong distributed-systems and software-engineering skills, and expertise in modern LLM optimization techniques.