CIS seminar series - Stéphane Lathuilière (Telecom Paris, France)

Centre for Intelligent Sensing

Self-supervised representation learning for video generation

(The Zoom link will be provided upon completion of the Eventbrite registration)Title: Self-supervised representation learning for video generation Presenter: Stéphane Lathuilière, Telecom Paris, France 
AbstractGenerating realistic images and videos has countless applications in different areas, ranging from photography technologies to e-commerce business. Recently, deep generative approaches have emerged as effective techniques for generation tasks. In this talk, we will illustrate how self-supervised representation learning can be employed to design video generation methods. First, we will present our recent framework for image animation. More precisely, our approach learns a motion representation that is employed to generate videos where an object in a source image is animated according to the motion of a driving video. In this task, we employ a motion representation based on keypoints that are learned in a self-supervised fashion. Therefore, our approach can animate any arbitrary object without using annotation or prior information about the specific object to animate. Then, we will introduce the unsupervised learning problem of playable video generation (PVG). In PVG, we aim at allowing a user to control the generated video by selecting a discrete action at every time step as when playing a video game. The difficulty of the task lies both in learning semantically consistent actions and in generating realistic videos conditioned on the user input. We introduce a novel framework for PVG which is trained in a self-supervised manner on a large dataset of unlabelled videos. We employ an encoder-decoder architecture where the predicted action labels act as bottleneck. The network is constrained to learn a rich action space using, as main driving loss, a reconstruction loss on the generated video. Finally, we will see how this approach can be extended to model 3D environments and unlock manipulation in space and time.BioStéphane Lathuilière is an associate professor (maître de conférence) at Telecom Paris, France, in the multimedia team. Until October 2019, he was a post-doctoral fellow at the University of Trento (Italy) in the Multimedia and Human Understanding Group, led by Prof. Nicu Sebe and Prof. Elisa Ricci. He received the M.Sc. degree in applied mathematics and computer science from ENSIMAG, Grenoble Institute of Technology (Grenoble INP), France, in 2014. He completed his master thesis at the International Research Institute MICA (Hanoi, Vietnam). He worked towards his Ph.D. in mathematics and computer science in the Perception Team at Inria under the supervision of Dr. Radu Horaud, and obtained it from Université Grenoble Alpes (France) in 2018. His research interests cover machine learning for computer vision problems (eg. domain adaptation, continual learning) and deep models for image and video generation. He regularly publishes papers in the most prestigious computer vision conferences (CVPR, ICCV, ECCV, NeurIPS) and top journals (IEEE TPAMI).Programme13:00 - 13:05: Opening and introduction13:05 - 13:50: Talk by Stéphane Lathuilière 13:50 - 14:00: Q&amp;A and closing remarksHost: Changjae Oh, Centre for Intelligent Sensing, Queen Mary University of London

Our {communityGuidelinesLink} describe the sort of content we prohibit on Eventbrite. If you suspect an event may be in violation, you can report it to us so we can investigate.

To report other categories of prohibited or illegal content, submit {link}

CIS seminar series - Stéphane Lathuilière (Telecom Paris, France)

More events from Centre for Intelligent Sensing

Discover more events from Centre for Intelligent Sensing, from Science & Tech to other experiences you might love.

CIS seminar series - Stéphane Lathuilière (Telecom Paris, France)

More events from Centre for Intelligent Sensing

Discover more events from Centre for Intelligent Sensing, from Science & Tech to other experiences you might love.

You might also like...

Browse more events with different dates, prices, and formats to find your next great experience.