LMVP: Video Predictor with Leaked Motion Information

by   Dong Wang, et al.
Duke University

We propose a Leaked Motion Video Predictor (LMVP) to predict future frames by capturing the spatial and temporal dependencies from given inputs. The motion is modeled by a newly proposed component, motion guider, which plays the role of both learner and teacher. Specifically, it learns the temporal features from real data and guides the generator to predict future frames. The spatial consistency in video is modeled by an adaptive filtering network. To further ensure the spatio-temporal consistency of the prediction, a discriminator is also adopted to distinguish the real and generated frames. Further, the discriminator leaks information to the motion guider and the generator to help the learning of motion. The proposed LMVP can effectively learn the static and temporal features in videos without the need for human labeling. Experiments on synthetic and real data demonstrate that LMVP can yield state-of-the-art results.


page 4

page 7


Convolutional Transformer based Dual Discriminator Generative Adversarial Networks for Video Anomaly Detection

Detecting abnormal activities in real-world surveillance videos is an im...

Two-stream Collaborative Learning with Spatial-Temporal Attention for Video Classification

Video classification is highly important with wide applications, such as...

Spatio-Temporal Branching for Motion Prediction using Motion Increments

Human motion prediction (HMP) has emerged as a popular research topic du...

To Create What You Tell: Generating Videos from Captions

We are creating multimedia contents everyday and everywhere. While autom...

VCT: A Video Compression Transformer

We show how transformers can be used to vastly simplify neural video com...

Spatio-Temporal Image Boundary Extrapolation

Boundary prediction in images as well as video has been a very active to...

Please sign up or login with your details

Forgot password? Click here to reset