Diverse Video Generation from a Single Video

05/11/2022
by   Niv Haim, et al.
18

GANs are able to perform generation and manipulation tasks, trained on a single video. However, these single video GANs require unreasonable amount of time to train on a single video, rendering them almost impractical. In this paper we question the necessity of a GAN for generation from a single video, and introduce a non-parametric baseline for a variety of generation and manipulation tasks. We revive classical space-time patches-nearest-neighbors approaches and adapt them to a scalable unconditional generative model, without any learning. This simple baseline surprisingly outperforms single-video GANs in visual quality and realism (confirmed by quantitative and qualitative evaluations), and is disproportionately faster (runtime reduced from several days to seconds). Our approach is easily scaled to Full-HD videos. We also use the same framework to demonstrate video analogies and spatio-temporal retargeting. These observations show that classical approaches significantly outperform heavy deep learning machinery for these tasks. This sets a new baseline for single-video generation and manipulation tasks, and no less important – makes diverse generation from a single video practically possible for the first time.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/17/2021

Diverse Generation from a Single Video Made Possible

Most advanced video generation and manipulation methods train on a large...
research
03/29/2021

Drop the GAN: In Defense of Patches Nearest Neighbors as Single Image Generative Models

Single image generative models perform synthesis and manipulation tasks ...
research
06/29/2022

3D-Aware Video Generation

Generative models have emerged as an essential building block for many i...
research
03/03/2022

Playable Environments: Video Manipulation in Space and Time

We present Playable Environments - a new representation for interactive ...
research
12/13/2022

PV3D: A 3D Generative Model for Portrait Video Generation

Recent advances in generative adversarial networks (GANs) have demonstra...
research
04/04/2020

Multi-Variate Temporal GAN for Large Scale Video Generation

In this paper, we present a network architecture for video generation th...
research
11/21/2020

Stochastic Talking Face Generation Using Latent Distribution Matching

The ability to envisage the visual of a talking face based just on heari...

Please sign up or login with your details

Forgot password? Click here to reset