DeepAI
Log In Sign Up

Sync-DRAW: Automatic Video Generation using Deep Recurrent Attentive Architectures

11/30/2016
by   Gaurav Mittal, et al.
0

This paper introduces a novel approach for generating videos called Synchronized Deep Recurrent Attentive Writer (Sync-DRAW). Sync-DRAW can also perform text-to-video generation which, to the best of our knowledge, makes it the first approach of its kind. It combines a Variational Autoencoder (VAE) with a Recurrent Attention Mechanism in a novel manner to create a temporally dependent sequence of frames that are gradually formed over time. The recurrent attention mechanism in Sync-DRAW attends to each individual frame of the video in sychronization, while the VAE learns a latent distribution for the entire video at the global level. Our experiments with Bouncing MNIST, KTH and UCF-101 suggest that Sync-DRAW is efficient in learning the spatial and temporal information of the videos and generates frames with high structural integrity, and can generate videos from simple captions on these datasets. (Accepted as oral paper in ACM-Multimedia 2017)

READ FULL TEXT

page 1

page 3

page 5

page 6

page 7

page 8

02/16/2015

DRAW: A Recurrent Neural Network For Image Generation

This paper introduces the Deep Recurrent Attentive Writer (DRAW) neural ...
10/01/2018

Where and When to Look? Spatio-temporal Attention for Action Recognition in Videos

Inspired by the observation that humans are able to process videos effic...
04/23/2018

To Create What You Tell: Generating Videos from Captions

We are creating multimedia contents everyday and everywhere. While autom...
03/03/2020

VQ-DRAW: A Sequential Discrete VAE

In this paper, I present VQ-DRAW, an algorithm for learning compact disc...
06/01/2018

Semi-Recurrent CNN-based VAE-GAN for Sequential Data Generation

A semi-recurrent hybrid VAE-GAN model for generating sequential data is ...
12/24/2020

An Efficient Recurrent Adversarial Framework for Unsupervised Real-Time Video Enhancement

Video enhancement is a challenging problem, more than that of stills, ma...
04/14/2021

Revisiting the Onsets and Frames Model with Additive Attention

Recent advances in automatic music transcription (AMT) have achieved hig...