DeepAI AI Chat
Log In Sign Up

Generating Videos with Scene Dynamics

by   Carl Vondrick, et al.
University of Maryland, Baltimore County

We capitalize on large amounts of unlabeled video in order to learn a model of scene dynamics for both video recognition tasks (e.g. action classification) and video generation tasks (e.g. future prediction). We propose a generative adversarial network for video with a spatio-temporal convolutional architecture that untangles the scene's foreground from the background. Experiments suggest this model can generate tiny videos up to a second at full frame rate better than simple baselines, and we show its utility at predicting plausible futures of static images. Moreover, experiments and visualizations show the model internally learns useful features for recognizing actions with minimal supervision, suggesting scene dynamics are a promising signal for representation learning. We believe generative video models can impact many applications in video understanding and simulation.


page 5

page 6

page 8


Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks

In the deep learning era, long video generation of high-quality still re...

V3GAN: Decomposing Background, Foreground and Motion for Video Generation

Video generation is a challenging task that requires modeling plausible ...

Generative Video Transformer: Can Objects be the Words?

Transformers have been successful for many natural language processing t...

Towards an Understanding of Our World by GANing Videos in the Wild

Existing generative video models work well only for videos with a static...

Towards Unsupervised Familiar Scene Recognition in Egocentric Videos

Nowadays, there is an upsurge of interest in using lifelogging devices. ...

VOS-GAN: Adversarial Learning of Visual-Temporal Dynamics for Unsupervised Dense Prediction in Videos

Recent GAN-based video generation approaches model videos as the combina...

Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition

Human activities often occur in specific scene contexts, e.g., playing b...

Code Repositories


Generating Videos with Scene Dynamics. NIPS 2016.

view repo


Generating Videos with Scene Dynamics

view repo