Recurrent Deconvolutional Generative Adversarial Networks with Application to Text Guided Video Generation

08/13/2020
by   Hongyuan Yu, et al.
15

This paper proposes a novel model for video generation and especially makes the attempt to deal with the problem of video generation from text descriptions, i.e., synthesizing realistic videos conditioned on given texts. Existing video generation methods cannot be easily adapted to handle this task well, due to the frame discontinuity issue and their text-free generation schemes. To address these problems, we propose a recurrent deconvolutional generative adversarial network (RD-GAN), which includes a recurrent deconvolutional network (RDN) as the generator and a 3D convolutional neural network (3D-CNN) as the discriminator. The RDN is a deconvolutional version of conventional recurrent neural network, which can well model the long-range temporal dependency of generated video frames and make good use of conditional information. The proposed model can be jointly trained by pushing the RDN to generate realistic videos so that the 3D-CNN cannot distinguish them from real ones. We apply the proposed RD-GAN to a series of tasks including conventional video generation, conditional video generation, video prediction and video classification, and demonstrate its effectiveness by achieving well performance.

READ FULL TEXT
research
02/21/2022

Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks

In the deep learning era, long video generation of high-quality still re...
research
09/04/2020

TiVGAN: Text to Image to Video Generation with Step-by-Step Evolutionary Generator

Advances in technology have led to the development of methods that can c...
research
07/29/2021

Video Generation from Text Employing Latent Path Construction for Temporal Modeling

Video generation is one of the most challenging tasks in Machine Learnin...
research
06/01/2018

Semi-Recurrent CNN-based VAE-GAN for Sequential Data Generation

A semi-recurrent hybrid VAE-GAN model for generating sequential data is ...
research
09/07/2021

Perceptual Learned Video Compression with Recurrent Conditional GAN

This paper proposes a Perceptual Learned Video Compression (PLVC) approa...
research
12/30/2022

Modified Query Expansion Through Generative Adversarial Networks for Information Extraction in E-Commerce

This work addresses an alternative approach for query expansion (QE) usi...
research
11/21/2016

Temporal Generative Adversarial Nets with Singular Value Clipping

In this paper, we propose a generative model, Temporal Generative Advers...

Please sign up or login with your details

Forgot password? Click here to reset