DeepAI AI Chat
Log In Sign Up

Generating Videos with Scene Dynamics

09/08/2016
by   Carl Vondrick, et al.
MIT
University of Maryland, Baltimore County
0

We capitalize on large amounts of unlabeled video in order to learn a model of scene dynamics for both video recognition tasks (e.g. action classification) and video generation tasks (e.g. future prediction). We propose a generative adversarial network for video with a spatio-temporal convolutional architecture that untangles the scene's foreground from the background. Experiments suggest this model can generate tiny videos up to a second at full frame rate better than simple baselines, and we show its utility at predicting plausible futures of static images. Moreover, experiments and visualizations show the model internally learns useful features for recognizing actions with minimal supervision, suggesting scene dynamics are a promising signal for representation learning. We believe generative video models can impact many applications in video understanding and simulation.

READ FULL TEXT

page 5

page 6

page 8

02/21/2022

Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks

In the deep learning era, long video generation of high-quality still re...
03/26/2022

V3GAN: Decomposing Background, Foreground and Motion for Video Generation

Video generation is a challenging task that requires modeling plausible ...
07/20/2021

Generative Video Transformer: Can Objects be the Words?

Transformers have been successful for many natural language processing t...
11/30/2017

Towards an Understanding of Our World by GANing Videos in the Wild

Existing generative video models work well only for videos with a static...
05/10/2019

Towards Unsupervised Familiar Scene Recognition in Egocentric Videos

Nowadays, there is an upsurge of interest in using lifelogging devices. ...
03/24/2018

VOS-GAN: Adversarial Learning of Visual-Temporal Dynamics for Unsupervised Dense Prediction in Videos

Recent GAN-based video generation approaches model videos as the combina...
12/11/2019

Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition

Human activities often occur in specific scene contexts, e.g., playing b...

Code Repositories

videogan

Generating Videos with Scene Dynamics. NIPS 2016.


view repo

videogan

Generating Videos with Scene Dynamics


view repo