Flow Guidance Deformable Compensation Network for Video Frame Interpolation

11/22/2022
by   Pengcheng Lei, et al.
0

Motion-based video frame interpolation (VFI) methods have made remarkable progress with the development of deep convolutional networks over the past years. While their performance is often jeopardized by the inaccuracy of flow map estimation, especially in the case of large motion and occlusion. In this paper, we propose a flow guidance deformable compensation network (FGDCN) to overcome the drawbacks of existing motion-based methods. FGDCN decomposes the frame sampling process into two steps: a flow step and a deformation step. Specifically, the flow step utilizes a coarse-to-fine flow estimation network to directly estimate the intermediate flows and synthesizes an anchor frame simultaneously. To ensure the accuracy of the estimated flow, a distillation loss and a task-oriented loss are jointly employed in this step. Under the guidance of the flow priors learned in step one, the deformation step designs a pyramid deformable compensation network to compensate for the missing details of the flow step. In addition, a pyramid loss is proposed to supervise the model in both the image and frequency domain. Experimental results show that the proposed algorithm achieves excellent performance on various datasets with fewer parameters.

READ FULL TEXT

page 1

page 7

page 8

page 9

page 10

page 11

research
04/04/2021

PDWN: Pyramid Deformable Warping Network for Video Interpolation

Video interpolation aims to generate a non-existent intermediate frame g...
research
05/10/2023

SENDD: Sparse Efficient Neural Depth and Deformation for Tissue Tracking

Deformable tracking and real-time estimation of 3D tissue motion is esse...
research
08/24/2020

Video Interpolation via Generalized Deformable Convolution

Video interpolation aims at increasing the frame rate of a given video b...
research
08/15/2021

Asymmetric Bilateral Motion Estimation for Video Frame Interpolation

We propose a novel video frame interpolation algorithm based on asymmetr...
research
03/12/2020

Deformation Flow Based Two-Stream Network for Lip Reading

Lip reading is the task of recognizing the speech content by analyzing m...
research
01/17/2021

Temporal Spatial-Adaptive Interpolation with Deformable Refinement for Electron Microscopic Images

Recently, flow-based methods have achieved promising success in video fr...
research
09/27/2022

Frame Interpolation for Dynamic Scenes with Implicit Flow Encoding

In this paper, we propose an algorithm to interpolate between a pair of ...

Please sign up or login with your details

Forgot password? Click here to reset