Conversational Modelling Research Engineer
AI Researcher develops and fine-tunes large multimodal models for real-time conversational avatars, modeling verbal/non-verbal behaviors with low latency. Requires PhD or equivalent, hands-on experience with VLMs, PyTorch, and deep learning.
About the job
Responsibilities
- Conduct research on Large Multimodal Models in the context of Conversational Avatars (e.g. Neural Avatars, Talking-Heads).
- Develop methods to model both verbal and non-verbal aspects of conversation, adapting and controlling avatar behavior in real time, with low-latency.
- Experiment with fine-tuning, adaptation, and conditioning techniques to make AudioVisual Multimodal Models more expressive, controllable, and task-specific.
- Partner with the Applied ML team to take research from prototype to production.
- Stay up to date with cutting-edge advancements.
Requirements
- PhD (or near completion) in a relevant field, or equivalent research experience.
- Hands-on experience with Large Multimodal Models and a strong foundation in generative (language) models (e.g. VQA, Audio/Video understanding, captioning, behavioral analysis, Translation, Speech to Speech systems).
- Experience in fine-tuning/adapting VLMs for control, conditioning, or downstream tasks.
- Solid background in deep learning and foundation models.
- Strong PyTorch skills and comfort building deep learning pipelines.
Nice-to-Haves
- Knowledge of large-scale model training and optimization.
- Experience in duplex-conversational models.
- Broader understanding of generative AI across modalities.
- Exposure to software development best practices.
- Flexible, experimental mindset (working across research and engineering).
- (Bonus) Publications at EMNLP, COLING, NeurIPS, ICLR, CVPR, ICCV.
Skills
PyTorch, Large Multimodal Models, Vision Language Models, Generative Models, Deep Learning, Fine-Tuning, Audio/Video Understanding, Vqa, Speech To Speech, Neural Avatars
Similar jobs
AI Research jobs
Improbable
Remote
Conduct applied research on AI agents, designing experiments and evaluation systems to improve reliability, context retention, and multi-step task completion. The role requires strong AI/ML research, engineering, experimental design, and communication skills.
Anthropic
San Francisco, CA
Research Engineer building large-scale AI capability evaluations, telemetry, data pipelines, and analysis tools for Anthropic’s Takeoff Intel team. The role requires hands-on large language model experimentation, rapid prototyping, data expertise, and strong research collaboration.
Sardine
United States
Conduct applied research on foundation models for fraud detection using large-scale behavioral and financial-risk data. The role spans experimentation, evaluation, production deployment, and cross-functional work on model governance, requiring 4+ years of applied ML experience and strong Python and SQL skills.
OpenAI
San Francisco, CA
Researcher or engineer focused on designing, evaluating, and productionizing oversight systems and safety mitigations for autonomous AI agents. The role requires strong systems or security reasoning, threat-modeling ability, and experience building practical evaluations and controls.
OpenAI
San Francisco, CA
Researcher focused on training and evaluating frontier AI agents, mining incidents, and building scalable safety measurement systems. The role requires strong research or ML engineering execution, quantitative judgment, and the ability to own ambiguous projects end to end.