Member of Technical Staff, Applied Research
Build and productionize vision-language models for document understanding at LlamaIndex. Focus on training, fine-tuning, synthetic data, benchmarking, and turning research prototypes into accurate, low-latency production systems for real-world PDFs, tables, and enterprise docs. Requires 3+ years ML engineering/applied research experience with strong PyTorch skills.
About the job
Responsibilities
- Develop and train vision-language models for document processing and document understanding.
- Build data pipelines for data curation, synthetic data generation, labeling, and benchmark creation.
- Evaluate base models and perform post-training or fine-tuning to hit specific performance targets.
- Improve model accuracy, latency, and cost-effectiveness across real-world document workflows.
- Design and maintain benchmarks to measure extraction quality, layout understanding, OCR performance, reasoning accuracy, and end-to-end system reliability.
- Work with messy real-world documents, including PDFs, scanned documents, tables, charts, forms, and multi-page enterprise documents.
- Collaborate with engineering to move successful research prototypes into production.
- Work directly with customers when needed to translate product requirements into benchmarks, experiments, and model improvements.
- Stay close to the latest research in vision-language models, document AI, post-training, synthetic data, and agentic systems.
- Use modern AI coding workflows and tools to move quickly.
Requirements
- 3–7 years of experience in machine learning engineering, applied research, or research engineering.
- Strong ML foundation, including hands-on experience benchmarking and training models.
- Strong Python skills and comfort with modern ML tooling, especially PyTorch.
- Experience with computer vision, vision-language models, NLP, document AI, OCR, extraction, or agentic AI systems.
- Ability to build experiments, evaluate results, and iterate quickly toward measurable performance improvements.
- Strong engineering judgment and ability to write clean, production-quality code.
- Comfort working in a fast-paced startup environment with high ownership and limited structure.
- Adaptable, scrappy, and self-directed — someone who can figure things out without waiting to be told.
- Strong technical writing and communication skills.
Nice-to-Haves
- Prior startup experience, especially at an early-stage or high-growth AI company.
- Experience as a founder or early startup engineer.
- Experience building or improving document processing systems.
- Experience with synthetic data generation, post-training, fine-tuning, or benchmark design.
- Familiarity with tools such as vLLM, Pydantic, uv, ruff, mypy, Claude Code, Cursor, or similar modern AI engineering workflows.
- Experience with open-source AI infrastructure or developer tools.
Skills
PyTorch, Python, Vision-Language Models, Computer Vision, Document Ai, Ocr, NLP, Synthetic Data Generation, Fine-Tuning, Benchmarking, vLLM
Similar jobs
ML Engineering jobs
Mirage
New York, NY
Build and deploy agentic systems that power AI-driven creative video workflows. The role requires 5+ years of experience, production ML or agentic pipeline development, context engineering, and expertise in evaluation and agent infrastructure.
Mirage
New York, NY
Build and advance agentic machine-learning systems for multimodal creative tasks, with a focus on video understanding, reasoning, control, and tool use. The role requires strong production ML or agent-pipeline experience and deep knowledge of modern LLM techniques.
Taste Labs
San Francisco, CA
Build evaluation methods, RL environments, agent tooling, and scalable infrastructure that make subjective qualities such as design and taste measurable for frontier AI models. The role requires experience with evaluations, RL environments, ML or post-training, plus strong backend engineering skills.
Mirage
New York, NY
Build and scale generative video and multimodal models, optimizing training and inference for efficiency, throughput, and ultra-low latency. The role requires deep learning systems expertise, strong PyTorch/CUDA experience, and the ability to move research models into production.
Earnin
Mountain View, CA
Build production-grade AI agents, evaluation infrastructure, and developer tooling that make AI-assisted engineering faster, safer, and reusable across teams. The role requires software engineering experience, platform or internal developer-product experience, and hands-on expertise with LLM integration and orchestration.