Log In Sign Up

Frame Interpolation for Dynamic Scenes with Implicit Flow Encoding

by   Pedro Figueirêdo, et al.

In this paper, we propose an algorithm to interpolate between a pair of images of a dynamic scene. While in the past years significant progress in frame interpolation has been made, current approaches are not able to handle images with brightness and illumination changes, which are common even when the images are captured shortly apart. We propose to address this problem by taking advantage of the existing optical flow methods that are highly robust to the variations in the illumination. Specifically, using the bidirectional flows estimated using an existing pre-trained flow network, we predict the flows from an intermediate frame to the two input images. To do this, we propose to encode the bidirectional flows into a coordinate-based network, powered by a hypernetwork, to obtain a continuous representation of the flow across time. Once we obtain the estimated flows, we use them within an existing blending network to obtain the final intermediate frame. Through extensive experiments, we demonstrate that our approach is able to produce significantly better results than state-of-the-art frame interpolation algorithms.


page 7

page 8

page 13

page 14

page 15

page 16

page 17

page 18


Video Interpolation by Event-driven Anisotropic Adjustment of Optical Flow

Video frame interpolation is a challenging task due to the ever-changing...

RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation

We propose RIFE, a Real-time Intermediate Flow Estimation algorithm for ...

Beyond a Video Frame Interpolator: A Space Decoupled Learning Approach to Continuous Image Transition

Video frame interpolation (VFI) aims to improve the temporal resolution ...

Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation

Given two consecutive frames, video interpolation aims at generating int...

Many-to-many Splatting for Efficient Video Frame Interpolation

Motion-based video frame interpolation commonly relies on optical flow t...

Flow Guidance Deformable Compensation Network for Video Frame Interpolation

Motion-based video frame interpolation (VFI) methods have made remarkabl...

Adaptive Future Frame Prediction with Ensemble Network

Future frame prediction in videos is a challenging problem because video...