LMVP: Video Predictor with Leaked Motion Information

06/24/2019
by   Dong Wang, et al.
Duke University
1

We propose a Leaked Motion Video Predictor (LMVP) to predict future frames by capturing the spatial and temporal dependencies from given inputs. The motion is modeled by a newly proposed component, motion guider, which plays the role of both learner and teacher. Specifically, it learns the temporal features from real data and guides the generator to predict future frames. The spatial consistency in video is modeled by an adaptive filtering network. To further ensure the spatio-temporal consistency of the prediction, a discriminator is also adopted to distinguish the real and generated frames. Further, the discriminator leaks information to the motion guider and the generator to help the learning of motion. The proposed LMVP can effectively learn the static and temporal features in videos without the need for human labeling. Experiments on synthetic and real data demonstrate that LMVP can yield state-of-the-art results.

READ FULL TEXT

page 4

page 7

07/29/2021

Convolutional Transformer based Dual Discriminator Generative Adversarial Networks for Video Anomaly Detection

Detecting abnormal activities in real-world surveillance videos is an im...
11/09/2017

Two-stream Collaborative Learning with Spatial-Temporal Attention for Video Classification

Video classification is highly important with wide applications, such as...
08/02/2023

Spatio-Temporal Branching for Motion Prediction using Motion Increments

Human motion prediction (HMP) has emerged as a popular research topic du...
04/23/2018

To Create What You Tell: Generating Videos from Captions

We are creating multimedia contents everyday and everywhere. While autom...
06/15/2022

VCT: A Video Compression Transformer

We show how transformers can be used to vastly simplify neural video com...
05/24/2016

Spatio-Temporal Image Boundary Extrapolation

Boundary prediction in images as well as video has been a very active to...

Please sign up or login with your details

Forgot password? Click here to reset