TiVGAN: Text to Image to Video Generation with Step-by-Step Evolutionary Generator

09/04/2020
by   Doyeon Kim, et al.
13

Advances in technology have led to the development of methods that can create desired visual multimedia. In particular, image generation using deep learning has been extensively studied across diverse fields. In comparison, video generation, especially on conditional inputs, remains a challenging and less explored area. To narrow this gap, we aim to train our model to produce a video corresponding to a given text description. We propose a novel training framework, Text-to-Image-to-Video Generative Adversarial Network (TiVGAN), which evolves frame-by-frame and finally produces a full-length video. In the first phase, we focus on creating a high-quality single video frame while learning the relationship between the text and an image. As the steps proceed, our model is trained gradually on more number of consecutive frames.This step-by-step learning process helps stabilize the training and enables the creation of high-resolution video based on conditional text descriptions. Qualitative and quantitative experimental results on various datasets demonstrate the effectiveness of the proposed method.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 8

page 9

page 10

research
08/13/2020

Recurrent Deconvolutional Generative Adversarial Networks with Application to Text Guided Video Generation

This paper proposes a novel model for video generation and especially ma...
research
09/01/2023

VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation

In this paper, we present VideoGen, a text-to-video generation approach,...
research
09/19/2021

ComicGAN: Text-to-Comic Generative Adversarial Network

Drawing and annotating comic illustrations is a complex and difficult pr...
research
05/07/2019

Spatially Constrained Generative Adversarial Networks for Conditional Image Generation

Image generation has raised tremendous attention in both academic and in...
research
10/01/2017

Video Generation From Text

Generating videos from text has proven to be a significant challenge for...
research
10/06/2022

Text-driven Video Prediction

Current video generation models usually convert signals indicating appea...
research
11/25/2022

TPA-Net: Generate A Dataset for Text to Physics-based Animation

Recent breakthroughs in Vision-Language (V L) joint research have achi...

Please sign up or login with your details

Forgot password? Click here to reset