Senior Machine Learning Engineer, Multimodal AI
Build and productionize multimodal AI systems that turn unstructured healthcare documents, faxes, and transcripts into reliable structured data and automated workflows using LLMs, OCR, and voice AI.
About the job
What You’ll Work On
- Build and improve multimodal AI pipelines that process healthcare documents, OCR output, transcripts, and workflow context into structured facts and decisions.
- Design LLM-powered extraction, classification, validation, and routing systems for operational and clinical workflows.
- Improve document intelligence systems across OCR, schema extraction, confidence scoring, error handling, and low-quality input recovery.
- Develop voice AI workflows for patient and provider outreach, transcript understanding, post-call extraction, and follow-up automation.
- Create evaluation harnesses, benchmarks, and regression tests for extraction quality, hallucination prevention, workflow accuracy, and model changes.
- Decide when to use LLMs, deterministic logic, retrieval, human review, or hybrid systems to maximize quality and reliability.
- Partner with product and engineering to identify the highest-leverage automation opportunities and translate them into shipped systems.
- Optimize cost, latency, and reliability across model providers and infrastructure layers.
- Work closely with backend engineers to deploy AI systems into our AWS and serverless environment with strong observability and operational rigor.
Technical Requirements
- Strong experience building production AI systems around LLMs, OCR, and unstructured data workflows.
- Proven track record shipping applied AI products, not just prototyping models offline.
- Deep familiarity with modern LLM workflows including prompting, structured outputs, tool use, retries, fallbacks, guardrails, and model evaluation.
- Experience with document intelligence systems such as OCR pipelines, document extraction, classification, post-processing, and confidence-based review flows.
- Experience with voice or conversational AI, or adjacent systems involving transcripts, call automation, and conversational extraction.
- Strong proficiency in Python and comfort working in production codebases with APIs, queues, and backend services.
- Experience deploying and operating AI systems in AWS or similar cloud environments, including serverless or event-driven architectures.
- Strong instincts around evaluation, benchmarking, monitoring, and quality assurance for real-world AI systems.
- Ability to work across structured and unstructured data and design systems that are robust to noisy, incomplete, and ambiguous inputs.
Nice to Have
- Experience in healthcare, claims, revenue cycle, or regulated operational environments.
- Experience with human-in-the-loop workflow design and review tooling.
- Familiarity with telephony vendors, speech systems, or conversational agent infrastructure.
- Experience comparing and routing across model providers such as OpenAI, Anthropic, Bedrock, or equivalent.
- Experience designing internal tools or operational systems used directly by workflow teams.
- Background in machine learning, applied NLP, information extraction, or related fields.
Skills
Python, LLMs, Ocr, AWS, Document Intelligence, Voice Ai, Prompt Engineering, Model Evaluation, Serverless, Structured Outputs
Similar jobs
ML Engineering jobs
Beacon Biosignals
United States
The Senior Algorithm Engineer leads development and production deployment of machine and deep learning algorithms for biosignal and medical-device applications. The role requires 5+ years of industry experience, DSP and statistics expertise, PyTorch proficiency, and familiarity with regulated health or similar domains.
Instacart
United StatesBuild and deploy machine learning systems that apply economic theory, econometrics, and causal inference to marketplace problems. The role requires advanced training in economics, strong Python and data skills, and production ML experience for senior-level hires.
Everlaw
Oakland, CA
Builds and operates AI platform capabilities including RAG pipelines, semantic retrieval, agentic orchestration, and LLM integrations to power legal tech products. Requires 4+ years in distributed cloud systems, AI/ML experience, and proficiency in modern programming languages.
Upstart
United States
Build and operate backend infrastructure for machine learning model training, serving, feature management, and marketplace simulation. The role requires 6+ years of software engineering experience, distributed systems expertise, and experience with production ML platforms.
Ai2
Seattle, WA
Senior Research Engineer tailoring and deploying machine learning models for partner applications across geospatial and environmental domains. The role requires PyTorch expertise, end-to-end ML deployment experience, geospatial tools knowledge, and strong independent execution.