Video Frame Interpolation Based on Deformable Kernel Region

04/25/2022
by   Haoyue Tian, et al.
30

Video frame interpolation task has recently become more and more prevalent in the computer vision field. At present, a number of researches based on deep learning have achieved great success. Most of them are either based on optical flow information, or interpolation kernel, or a combination of these two methods. However, these methods have ignored that there are grid restrictions on the position of kernel region during synthesizing each target pixel. These limitations result in that they cannot well adapt to the irregularity of object shape and uncertainty of motion, which may lead to irrelevant reference pixels used for interpolation. In order to solve this problem, we revisit the deformable convolution for video interpolation, which can break the fixed grid restrictions on the kernel region, making the distribution of reference points more suitable for the shape of the object, and thus warp a more accurate interpolation frame. Experiments are conducted on four datasets to demonstrate the superior performance of the proposed model in comparison to the state-of-the-art alternatives.

READ FULL TEXT

page 3

page 6

page 9

page 10

research
08/24/2020

Video Interpolation via Generalized Deformable Convolution

Video interpolation aims at increasing the frame rate of a given video b...
research
07/24/2019

Learning Spatial Transform for Video Frame Interpolation

Video frame interpolation is one of the most challenging tasks in the vi...
research
06/04/2017

Deep Frame Interpolation

This work presents a supervised learning based approach to the computer ...
research
10/20/2018

MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement

Motion estimation (ME) and motion compensation (MC) have been widely use...
research
04/07/2022

Many-to-many Splatting for Efficient Video Frame Interpolation

Motion-based video frame interpolation commonly relies on optical flow t...
research
10/30/2020

LIFI: Towards Linguistically Informed Frame Interpolation

In this work, we explore a new problem of frame interpolation for speech...
research
07/31/2023

Uncertainty-Guided Spatial Pruning Architecture for Efficient Frame Interpolation

The video frame interpolation (VFI) model applies the convolution operat...

Please sign up or login with your details

Forgot password? Click here to reset