V3GAN: Decomposing Background, Foreground and Motion for Video Generation

03/26/2022
by   Arti Keshari, et al.
0

Video generation is a challenging task that requires modeling plausible spatial and temporal dynamics in a video. Inspired by how humans perceive a video by grouping a scene into moving and stationary components, we propose a method that decomposes the task of video generation into the synthesis of foreground, background and motion. Foreground and background together describe the appearance, whereas motion specifies how the foreground moves in a video over time. We propose V3GAN, a novel three-branch generative adversarial network where two branches model foreground and background information, while the third branch models the temporal information without any supervision. The foreground branch is augmented with our novel feature-level masking layer that aids in learning an accurate mask for foreground and background separation. To encourage motion consistency, we further propose a shuffling loss for the video discriminator. Extensive quantitative and qualitative analysis on synthetic as well as real-world benchmark datasets demonstrates that V3GAN outperforms the state-of-the-art methods by a significant margin.

READ FULL TEXT

page 5

page 15

page 17

page 18

page 19

page 20

page 22

page 23

research
03/28/2021

ACSNet: Action-Context Separation Network for Weakly Supervised Temporal Action Localization

The object of Weakly-supervised Temporal Action Localization (WS-TAL) is...
research
12/04/2015

Motion trails from time-lapse video

From an image sequence captured by a stationary camera, background subtr...
research
11/24/2021

Layered Controllable Video Generation

We introduce layered controllable video generation, where we, without an...
research
12/12/2019

To See in the Dark: N2DGAN for Background Modeling in Nighttime Scene

Due to the deteriorated conditions of lack and uneven lighting, nightti...
research
10/02/2022

Cross-identity Video Motion Retargeting with Joint Transformation and Synthesis

In this paper, we propose a novel dual-branch Transformation-Synthesis n...
research
04/20/2019

Cubic LSTMs for Video Prediction

Predicting future frames in videos has become a promising direction of r...
research
06/27/2019

Automatically Extract the Semi-transparent Motion-blurred Hand from a Single Image

When we use video chat, video game, or other video applications, motion-...

Please sign up or login with your details

Forgot password? Click here to reset