Deep Animation Video Interpolation in the Wild

by   Li Siyao, et al.

In the animation industry, cartoon videos are usually produced at low frame rate since hand drawing of such frames is costly and time-consuming. Therefore, it is desirable to develop computational models that can automatically interpolate the in-between animation frames. However, existing video interpolation methods fail to produce satisfying results on animation data. Compared to natural videos, animation videos possess two unique characteristics that make frame interpolation difficult: 1) cartoons comprise lines and smooth color pieces. The smooth areas lack textures and make it difficult to estimate accurate motions on animation videos. 2) cartoons express stories via exaggeration. Some of the motions are non-linear and extremely large. In this work, we formally define and study the animation video interpolation problem for the first time. To address the aforementioned challenges, we propose an effective framework, AnimeInterp, with two dedicated modules in a coarse-to-fine manner. Specifically, 1) Segment-Guided Matching resolves the "lack of textures" challenge by exploiting global matching among color pieces that are piece-wise coherent. 2) Recurrent Flow Refinement resolves the "non-linear and extremely large motion" challenge by recurrent predictions using a transformer-like architecture. To facilitate comprehensive training and evaluations, we build a large-scale animation triplet dataset, ATD-12K, which comprises 12,000 triplets with rich annotations. Extensive experiments demonstrate that our approach outperforms existing state-of-the-art interpolation methods for animation videos. Notably, AnimeInterp shows favorable perceptual quality and robustness for animation scenarios in the wild. The proposed dataset and code are available at


page 1

page 2

page 3

page 4

page 7

page 8


H-VFI: Hierarchical Frame Interpolation for Videos with Large Motions

Capitalizing on the rapid development of neural networks, recent video f...

A Perceptual Quality Metric for Video Frame Interpolation

Research on video frame interpolation has made significant progress in r...

Render In-between: Motion Guided Video Synthesis for Action Interpolation

Upsampling videos of human activity is an interesting yet challenging ta...

Beyond Natural Motion: Exploring Discontinuity for Video Frame Interpolation

Video interpolation is the task that synthesizes the intermediate frame ...

Spatio-Temporal Multi-Flow Network for Video Frame Interpolation

Video frame interpolation (VFI) is currently a very active research topi...

JNMR: Joint Non-linear Motion Regression for Video Frame Interpolation

Video frame interpolation (VFI) aims to generate predictive frames by wa...

Video Stitching for Linear Camera Arrays

Despite the long history of image and video stitching research, existing...

Please sign up or login with your details

Forgot password? Click here to reset