Senior Applied Research Scientist - Foundation Models
Develops and optimizes transformer-based vision-language models for physical security, owning full-cycle training, fine-tuning, and deployment optimization. Collaborates cross-functionally to integrate models into the platform using PyTorch/TensorFlow and advanced AI techniques.
About the job
What you'll do
- Develop & Optimize VLMs: Design and optimize transformer-based vision-language models to understand images, videos, and text, and optimize for real-time inference.
- Pre-training & Fine-tuning: Own the full training pipeline—from pre-training on image-text data to fine-tuning for Ambient.ai’s physical security domain and use cases.
- Model Compression & Optimization: Apply techniques like distillation, quantization, and pruning to reduce model size and latency, enabling efficient edge deployment.
- Leverage Open-Source & Innovate: Use and extend state-of-the-art open-source models. Prototype new architectures and training methods to advance Ambient.ai’s multimodal AI research.
- Cross-Team Collaboration: Work with engineering and product teams to integrate models into the platform. Iterate based on real-world feedback and deployment data to improve performance.
- Research and Experimentation: Stay current with vision, NLP, and multimodal AI research. Design experiments to test new algorithms and continually enhance our core AI systems.
What you'll bring
- Ph.D. or Master’s in CS, EE, or related field, with a strong foundation in AI/ML (Ph.D. preferred or Master’s with strong experience)
- Proficient in Python/C++ and deep learning frameworks like PyTorch or TensorFlow. Comfortable with large-scale training pipelines
- Hands-on experience with CNNs, Transformers, and Vision Transformers (ViT). Strong understanding of vision-language models and how to fine-tune or adapt them
- Proven skills in model training and optimization, including fine-tuning on large datasets and applying distillation, quantization, or similar techniques. Experience with foundation or multimodal models is a plus.
- Strong problem-solving ability: quick prototyping, diagnosing failure cases, and iterating on solutions
- Startup experience preferred: Comfortable with ambiguity, fast iteration, and owning projects end-to-end
Skills
PyTorch, TensorFlow, Python, C++, Transformers, Vision Transformers, Cnns, Vision-Language Models, Model Distillation, Quantization, Model Pruning
Similar jobs
ML Engineering jobs
Databricks
United States
Build and productionize generative AI applications for U.S. federal customers, advise clients, and influence product direction. The role requires extensive data science and machine learning deployment experience, a graduate quantitative degree or equivalent experience, and U.S. security clearance eligibility.
Swayable
New York, NYSenior engineer developing and productizing AI, machine learning, scientific computing, and data-analysis capabilities for a high-performance analytics engine. Requires 5+ years building quantitative data-intensive software and expertise in Python, machine learning, scalable architecture, and distributed computing.
Mem0
San Francisco, CA
Own the end-to-end lifecycle of memory features for AI agents. Fine-tune models, implement research, build evaluations, and ship production systems with Engineering.
PrizePicks
United States
Build and ship production Applied AI capabilities, including agent infrastructure, RAG services, evaluation systems, and AI-powered engineering workflows. The role requires 6+ years of software engineering experience, strong backend and distributed-systems skills, and direct experience delivering LLM- or ML-powered products.
Ai2
Seattle, WA
Senior Research Engineer tailoring and deploying machine learning models for partner applications across geospatial and environmental domains. The role requires PyTorch expertise, end-to-end ML deployment experience, geospatial tools knowledge, and strong independent execution.