JNMR: Joint Non-linear Motion Regression for Video Frame Interpolation

06/09/2022
by   Meiqin Liu, et al.
0

Video frame interpolation (VFI) aims to generate predictive frames by warping learnable motions from the bidirectional historical references. Most existing works utilize spatio-temporal semantic information extractor to realize motion estimation and interpolation modeling, not enough considering with the real mechanistic rationality of generated middle motions. In this paper, we reformulate VFI as a multi-variable non-linear (MNL) regression problem, and a Joint Non-linear Motion Regression (JNMR) strategy is proposed to model complicated motions of inter-frame. To establish the MNL regression, ConvLSTM is adopted to construct the distribution of complete motions in temporal dimension. The motion correlations between the target frame and multiple reference frames can be regressed by the modeled distribution. Moreover, the feature learning network is designed to optimize for the MNL regression modeling. A coarse-to-fine synthesis enhancement module is further conducted to learn visual dynamics at different resolutions through repetitive regression and interpolation. Highly competitive experimental results on frame interpolation show that the effectiveness and significant improvement compared with state-of-the-art performance, and the robustness of complicated motion estimation is improved by the MNL motion regression.

READ FULL TEXT

page 1

page 4

page 7

page 8

page 9

page 10

research
01/27/2022

Non-linear Motion Estimation for Video Frame Interpolation using Space-time Convolutions

Video frame interpolation aims to synthesize one or multiple frames betw...
research
12/15/2020

FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation

A majority of approaches solve the problem of video frame interpolation ...
research
03/29/2022

Long-term Video Frame Interpolation via Feature Propagation

Video frame interpolation (VFI) works generally predict intermediate fra...
research
06/10/2023

Shuffled Autoregression For Motion Interpolation

This work aims to provide a deep-learning solution for the motion interp...
research
06/14/2021

TimeLens: Event-based Video Frame Interpolation

State-of-the-art frame interpolation methods generate intermediate frame...
research
04/21/2022

Unsupervised Video Interpolation by Learning Multilayered 2.5D Motion Fields

The problem of video frame interpolation is to increase the temporal res...
research
04/06/2021

Deep Animation Video Interpolation in the Wild

In the animation industry, cartoon videos are usually produced at low fr...

Please sign up or login with your details

Forgot password? Click here to reset