A Two-Stream Variational Adversarial Network for Video Generation

12/03/2018
by   Ximeng Sun, et al.
0

Video generation is an inherently challenging task, as it requires the model to generate realistic content and motion simultaneously. Existing methods generate both motion and content together using a single generator network, but this approach may fail on complex videos. In this paper, we propose a two-stream video generation model that separates content and motion generation into two parallel generators, called Two-Stream Variational Adversarial Network (TwoStreamVAN). Our model outputs a realistic video given an input action label by progressively generating and fusing motion and content features at multiple scales using adaptive motion kernels. In addition, to better evaluate video generation models, we design a new synthetic human action dataset to bridge the difficulty gap between over-complicated human action datasets and simple toy datasets. Our model significantly outperforms existing methods on the standard Weizmann Human Action and MUG Facial Expression datasets, as well as our new dataset.

READ FULL TEXT

page 1

page 3

page 7

page 8

page 13

page 14

page 15

page 16

research
12/11/2019

G^3AN: This video does not exist. Disentangling motion and appearance for video generation

Creating realistic human videos introduces the challenge of being able t...
research
10/12/2019

Generating Human Action Videos by Coupling 3D Game Engines and Probabilistic Graphical Models

Deep video action recognition models have been highly successful in rece...
research
12/28/2022

NeMo: 3D Neural Motion Fields from Multiple Video Instances of the Same Action

The task of reconstructing 3D human motion has wideranging applications....
research
12/18/2021

Adversarial Memory Networks for Action Prediction

Action prediction aims to infer the forthcoming human action with partia...
research
07/07/2018

Video Prediction with Appearance and Motion Conditions

Video prediction aims to generate realistic future frames by learning dy...
research
12/04/2018

Conditional Video Generation Using Action-Appearance Captions

The field of automatic video generation has received a boost thanks to t...
research
06/14/2022

Recurrent Transformer Variational Autoencoders for Multi-Action Motion Synthesis

We consider the problem of synthesizing multi-action human motion sequen...

Please sign up or login with your details

Forgot password? Click here to reset