Diverse Video Generation using a Gaussian Process Trigger

07/09/2021
by   Gaurav Shrivastava, et al.
4

Generating future frames given a few context (or past) frames is a challenging task. It requires modeling the temporal coherence of videos and multi-modality in terms of diversity in the potential future states. Current variational approaches for video generation tend to marginalize over multi-modal future outcomes. Instead, we propose to explicitly model the multi-modality in the future outcomes and leverage it to sample diverse futures. Our approach, Diverse Video Generator, uses a Gaussian Process (GP) to learn priors on future states given the past and maintains a probability distribution over possible futures given a particular sample. In addition, we leverage the changes in this distribution over time to control the sampling of diverse future states by estimating the end of ongoing sequences. That is, we use the variance of GP over the output function space to trigger a change in an action sequence. We achieve state-of-the-art results on diverse future frame generation in terms of reconstruction quality and diversity of the generated sequences.

READ FULL TEXT

page 2

page 9

page 16

page 17

research
02/21/2018

Stochastic Video Generation with a Learned Prior

Generating video frames that accurately predict future world states is c...
research
10/15/2019

Neural Approximation of an Auto-Regressive Process through Confidence Guided Sampling

We propose a generic confidence-based approximation that can be plugged ...
research
11/30/2020

RegFlow: Probabilistic Flow-based Regression for Future Prediction

Predicting future states or actions of a given system remains a fundamen...
research
08/22/2019

Compositional Video Prediction

We present an approach for pixel-level future prediction given an input ...
research
04/05/2019

Point-to-Point Video Generation

While image manipulation achieves tremendous breakthroughs (e.g., genera...
research
06/20/2018

Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objective

For autonomous agents to successfully operate in the real world, anticip...
research
06/28/2012

A Multi-State Power Model for Adequacy Assessment of Distributed Generation via Universal Generating Function

The current and future developments of electric power systems are pushin...

Please sign up or login with your details

Forgot password? Click here to reset