DeepAI AI Chat
Log In Sign Up

Video Generation from Single Semantic Label Map

by   Junting Pan, et al.

This paper proposes the novel task of video generation conditioned on a SINGLE semantic label map, which provides a good balance between flexibility and quality in the generation process. Different from typical end-to-end approaches, which model both scene content and dynamics in a single step, we propose to decompose this difficult task into two sub-problems. As current image generation methods do better than video generation in terms of detail, we synthesize high quality content by only generating the first frame. Then we animate the scene based on its semantic meaning to obtain the temporally coherent video, giving us excellent results overall. We employ a cVAE for predicting optical flow as a beneficial intermediate step to generate a video sequence conditioned on the initial single frame. A semantic label map is integrated into the flow prediction module to achieve major improvements in the image-to-video generation process. Extensive experiments on the Cityscapes dataset show that our method outperforms all competing methods.


page 1

page 2

page 3

page 4

page 5

page 6

page 8

page 9


DTVNet: Dynamic Time-lapse Video Generation via Single Still Image

This paper presents a novel end-to-end dynamic time-lapse video generati...

Video Interpolation by Event-driven Anisotropic Adjustment of Optical Flow

Video frame interpolation is a challenging task due to the ever-changing...

Surveillance Video Parsing with Single Frame Supervision

Surveillance video parsing, which segments the video frames into several...

Learning Deep Video Stabilization without Optical Flow

Learning the necessary high-level reasoning for video stabilization with...

HSTR-Net: High Spatio-Temporal Resolution Video Generation For Wide Area Surveillance

Wide area surveillance has many applications and tracking of objects und...

Semantic Editing On Segmentation Map Via Multi-Expansion Loss

Semantic editing on segmentation map has been proposed as an intermediat...

Example-Guided Scene Image Synthesis using Masked Spatial-Channel Attention and Patch-Based Self-Supervision

Example-guided image synthesis has been recently attempted to synthesize...

Code Repositories


Pytorch implementation of "Video Generation from Single Semantic Label Map", CVPR 2019

view repo


Video Generation from Single Semantic Label Map

view repo