Label-Conditioned Next-Frame Video Generation with Neural Flows

by   David Donahue, et al.

Recent state-of-the-art video generation systems employ Generative Adversarial Networks (GANs) or Variational Autoencoders (VAEs) to produce novel videos. However, VAE models typically produce blurry outputs when faced with sub-optimal conditioning of the input, and GANs are known to be unstable for large output sizes. In addition, the output videos of these models are difficult to evaluate, partly because the GAN loss function is not an accurate measure of convergence. In this work, we propose using a state-of-the-art neural flow generator called Glow to generate videos conditioned on a textual label, one frame at a time. Neural flow models are more stable than standard GANs, as they only optimize a single cross entropy loss function, which is monotonic and avoids the circular convergence issues of the GAN minimax objective. In addition, we also show how to condition Glow on external context, while still preserving the invertible nature of each "flow" layer. Finally, we evaluate the proposed Glow model by calculating cross entropy on a held-out validation set of videos, in order to compare multiple versions of the proposed model via an ablation study. We show generated videos and discuss future improvements.


page 4

page 6

page 9


Rényi Generative Adversarial Networks

We propose a loss function for generative adversarial networks (GANs) us...

Least Squares Generative Adversarial Networks

Unsupervised learning with generative adversarial networks (GANs) has pr...

Conditional Video Generation Using Action-Appearance Captions

The field of automatic video generation has received a boost thanks to t...

JR-GAN: Jacobian Regularization for Generative Adversarial Networks

Generative adversarial networks (GANs) are notoriously difficult to trai...

Towards High Resolution Video Generation with Progressive Growing of Sliced Wasserstein GANs

The extension of image generation to video generation turns out to be a ...

Incremental Classifier Learning with Generative Adversarial Networks

In this paper, we address the incremental classifier learning problem, w...

Please sign up or login with your details

Forgot password? Click here to reset