Conversational Modelling Research Engineer

Sourced from flexgen.zya.me

Flexgen is seeking an AI Research Engineer to join its core AI team, focusing on foundation multimodal conversational models. The role involves conducting research on Large Multimodal Models for conversational avatars, developing methods to model verbal and non-verbal conversation, and taking research from prototype to production. Requires a PhD or equivalent research experience, strong PyTorch skills, and experience with VLMs.

• Conduct research on Large Multimodal Models in the context of Conversational Avatars (e.g. Neural Avatars, Talking-Heads)
• Develop methods to model both verbal and non-verbal aspects of conversation, adapting and controlling avatar behavior in real time, with low-latency
• Experiment with fine-tuning, adaptation, and conditioning techniques to make AudioVisual Multimodal Models more expressive, controllable, and task-specific
• Partner with the Applied ML team to take research from prototype to production
• Stay up to date with cutting-edge advancements — and help define what comes next

• A PhD (or near completion) in a relevant field, or equivalent research experience
• Hands-on experience with Large Multimodal Models and a strong foundation in generative (language) models (e.g. VQA, audio/video understanding, captioning, translation, speech-to-speech)
• Experience in fine-tuning/adapting VLMs for control, conditioning, or downstream tasks
• Solid background in deep learning and foundation models
• Strong PyTorch skills and comfort building deep learning pipelines

• Knowledge of large-scale model training and optimization
• Experience in duplex-conversational models
• Broader understanding of generative AI across modalities
• Exposure to software development best practices
• A flexible, experimental mindset comfortable working across research and engineering
• Publications at EMNLP, COLING, NeurIPS, ICLR, CVPR, or ICCV