Log In Sign Up

Stochastic Latent Residual Video Prediction

by   Jean-Yves Franceschi, et al.

Designing video prediction models that account for the inherent uncertainty of the future is challenging. Most works in the literature are based on stochastic image-autoregressive recurrent networks, which raises several performance and applicability issues. An alternative is to use fully latent temporal models which untie frame synthesis and temporal dynamics. However, no such model for stochastic video prediction has been proposed in the literature yet, due to design and training difficulties. In this paper, we overcome these difficulties by introducing a novel stochastic temporal model whose dynamics are governed in a latent space by a residual update rule. This first-order scheme is motivated by discretization schemes of differential equations. It naturally models video dynamics as it allows our simpler, more interpretable, latent model to outperform prior state-of-the-art methods on challenging datasets.


page 21

page 22

page 23

page 24

page 26

page 27

page 28

page 29


Wildfire Forecasting with Satellite Images and Deep Generative Model

Wildfire forecasting has been one of the most critical tasks that humani...

Latent Neural Differential Equations for Video Generation

Generative Adversarial Networks have recently shown promise for video ge...

Simple Video Generation using Neural ODEs

Despite having been studied to a great extent, the task of conditional g...

Taylor saves for later: disentanglement for video prediction using Taylor representation

Video prediction is a challenging task with wide application prospects i...

StretchBEV: Stretching Future Instance Prediction Spatially and Temporally

In self-driving, predicting future in terms of location and motion of al...

Prediction Under Uncertainty with Error-Encoding Networks

In this work we introduce a new framework for performing temporal predic...