DeepAI
Log In Sign Up

Frame Interpolation for Dynamic Scenes with Implicit Flow Encoding

09/27/2022
by   Pedro Figueirêdo, et al.
0

In this paper, we propose an algorithm to interpolate between a pair of images of a dynamic scene. While in the past years significant progress in frame interpolation has been made, current approaches are not able to handle images with brightness and illumination changes, which are common even when the images are captured shortly apart. We propose to address this problem by taking advantage of the existing optical flow methods that are highly robust to the variations in the illumination. Specifically, using the bidirectional flows estimated using an existing pre-trained flow network, we predict the flows from an intermediate frame to the two input images. To do this, we propose to encode the bidirectional flows into a coordinate-based network, powered by a hypernetwork, to obtain a continuous representation of the flow across time. Once we obtain the estimated flows, we use them within an existing blending network to obtain the final intermediate frame. Through extensive experiments, we demonstrate that our approach is able to produce significantly better results than state-of-the-art frame interpolation algorithms.

READ FULL TEXT

page 7

page 8

page 13

page 14

page 15

page 16

page 17

page 18

08/19/2022

Video Interpolation by Event-driven Anisotropic Adjustment of Optical Flow

Video frame interpolation is a challenging task due to the ever-changing...
11/12/2020

RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation

We propose RIFE, a Real-time Intermediate Flow Estimation algorithm for ...
03/18/2022

Beyond a Video Frame Interpolator: A Space Decoupled Learning Approach to Continuous Image Transition

Video frame interpolation (VFI) aims to improve the temporal resolution ...
11/30/2017

Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation

Given two consecutive frames, video interpolation aims at generating int...
04/07/2022

Many-to-many Splatting for Efficient Video Frame Interpolation

Motion-based video frame interpolation commonly relies on optical flow t...
11/22/2022

Flow Guidance Deformable Compensation Network for Video Frame Interpolation

Motion-based video frame interpolation (VFI) methods have made remarkabl...
11/13/2020

Adaptive Future Frame Prediction with Ensemble Network

Future frame prediction in videos is a challenging problem because video...