Learning Fine-Grained Motion Embedding for Landscape Animation

by   Hongwei Xue, et al.

In this paper we focus on landscape animation, which aims to generate time-lapse videos from a single landscape image. Motion is crucial for landscape animation as it determines how objects move in videos. Existing methods are able to generate appealing videos by learning motion from real time-lapse videos. However, current methods suffer from inaccurate motion generation, which leads to unrealistic video results. To tackle this problem, we propose a model named FGLA to generate high-quality and realistic videos by learning Fine-Grained motion embedding for Landscape Animation. Our model consists of two parts: (1) a motion encoder which embeds time-lapse motion in a fine-grained way. (2) a motion generator which generates realistic motion to animate input images. To train and evaluate on diverse time-lapse videos, we build the largest high-resolution Time-lapse video dataset with Diverse scenes, namely Time-lapse-D, which includes 16,874 video clips with over 10 million frames. Quantitative and qualitative experimental results demonstrate the superiority of our method. In particular, our method achieves relative improvements by 19 methods on our dataset. A user study carried out with 700 human subjects shows that our approach visually outperforms existing methods by a large margin.


page 1

page 3

page 5

page 8


DTVNet: Dynamic Time-lapse Video Generation via Single Still Image

This paper presents a novel end-to-end dynamic time-lapse video generati...

DeepLandscape: Adversarial Modeling of Landscape Video

We build a new model of landscape videos that can be trained on a mixtur...

Unsupervised Object-Level Video Summarization with Online Motion Auto-Encoder

Unsupervised video summarization plays an important role on digesting, b...

Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks

Taking a photo outside, can we predict the immediate future, e.g., how w...

StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Video

Face reenactment methods attempt to restore and re-animate portrait vide...

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking

We introduce PointOdyssey, a large-scale synthetic dataset, and data gen...

Exposing Deep-faked Videos by Anomalous Co-motion Pattern Detection

Recent deep learning based video synthesis approaches, in particular wit...

Please sign up or login with your details

Forgot password? Click here to reset