DeepAI
Log In Sign Up

RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation

11/12/2020
by   Zhewei Huang, et al.
8

We propose RIFE, a Real-time Intermediate Flow Estimation algorithm for Video Frame Interpolation (VFI). Most existing methods first estimate the bi-directional optical flows and then linearly combine them to approximate intermediate flows, leading to artifacts on motion boundaries. RIFE uses a neural network named IFNet that can directly estimate the intermediate flows from images. With the more precise flows and our simplified fusion process, RIFE can improve interpolation quality and have much better speed. Based on our proposed leakage distillation loss, RIFE can be trained in an end-to-end fashion. Experiments demonstrate that our method is significantly faster than existing VFI methods and can achieve state-of-the-art performance on public benchmarks. The code is available at https://github.com/hzwer/arXiv2020-RIFE.

READ FULL TEXT

page 2

page 4

page 6

page 8

11/30/2017

Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation

Given two consecutive frames, video interpolation aims at generating int...
05/27/2021

FastRIFE: Optimization of Real-Time Intermediate Flow Estimation for Video Frame Interpolation

The problem of video inter-frame interpolation is an essential task in t...
03/18/2022

Beyond a Video Frame Interpolator: A Space Decoupled Learning Approach to Continuous Image Transition

Video frame interpolation (VFI) aims to improve the temporal resolution ...
07/14/2022

Neighbor Correspondence Matching for Flow-based Video Frame Synthesis

Video frame synthesis, which consists of interpolation and extrapolation...
05/29/2022

IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation

Prevailing video frame interpolation algorithms, that generate the inter...
09/27/2022

Frame Interpolation for Dynamic Scenes with Implicit Flow Encoding

In this paper, we propose an algorithm to interpolate between a pair of ...
11/30/2021

Spatio-Temporal Multi-Flow Network for Video Frame Interpolation

Video frame interpolation (VFI) is currently a very active research topi...

Code Repositories

arXiv2020-RIFE

RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation


view repo

GIMP-ML

AI for GNU Image Manipulation Program


view repo

rife-ncnn-vulkan

RIFE, Real-Time Intermediate Flow Estimation for Video Frame Interpolation implemented with ncnn library


view repo

rife-ncnn-vulkan-python

A Python FFI of nihui/rife-ncnn-vulkan achieved with SWIG


view repo

media-vision

🎥 Visual media restoration using deep-learning-based colorization, upscaling, interpolation, and more.


view repo