Video Frame Interpolation with Densely Queried Bilateral Correlation

04/26/2023
by   Chang Zhou, et al.
0

Video Frame Interpolation (VFI) aims to synthesize non-existent intermediate frames between existent frames. Flow-based VFI algorithms estimate intermediate motion fields to warp the existent frames. Real-world motions' complexity and the reference frame's absence make motion estimation challenging. Many state-of-the-art approaches explicitly model the correlations between two neighboring frames for more accurate motion estimation. In common approaches, the receptive field of correlation modeling at higher resolution depends on the motion fields estimated beforehand. Such receptive field dependency makes common motion estimation approaches poor at coping with small and fast-moving objects. To better model correlations and to produce more accurate motion fields, we propose the Densely Queried Bilateral Correlation (DQBC) that gets rid of the receptive field dependency problem and thus is more friendly to small and fast-moving objects. The motion fields generated with the help of DQBC are further refined and up-sampled with context features. After the motion fields are fixed, a CNN-based SynthNet synthesizes the final interpolated frame. Experiments show that our approach enjoys higher accuracy and less inference time than the state-of-the-art. Source code is available at https://github.com/kinoud/DQBC.

READ FULL TEXT

page 3

page 6

page 7

research
04/05/2023

BiFormer: Learning Bilateral Motion Estimation via Bilateral Transformer for 4K Video Frame Interpolation

A novel 4K video frame interpolator based on bilateral transformer (BiFo...
research
08/15/2021

Asymmetric Bilateral Motion Estimation for Video Frame Interpolation

We propose a novel video frame interpolation algorithm based on asymmetr...
research
05/27/2021

FastRIFE: Optimization of Real-Time Intermediate Flow Estimation for Video Frame Interpolation

The problem of video inter-frame interpolation is an essential task in t...
research
05/29/2022

IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation

Prevailing video frame interpolation algorithms, that generate the inter...
research
09/09/2022

Sparsity-guided Network Design for Frame Interpolation

DNN-based frame interpolation, which generates intermediate frames from ...
research
01/18/2022

Motion Inbetweening via Deep Δ-Interpolator

We show that the task of synthesizing missing middle frames, commonly kn...
research
08/09/2023

Objects do not disappear: Video object detection by single-frame object location anticipation

Objects in videos are typically characterized by continuous smooth motio...

Please sign up or login with your details

Forgot password? Click here to reset