FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

12/06/2016
by   Eddy Ilg, et al.
0

The FlowNet demonstrated that optical flow estimation can be cast as a learning problem. However, the state of the art with regard to the quality of the flow has still been defined by traditional methods. Particularly on small displacements and real-world data, FlowNet cannot compete with variational methods. In this paper, we advance the concept of end-to-end learning of optical flow and make it work really well. The large improvements in quality and speed are caused by three major contributions: first, we focus on the training data and show that the schedule of presenting data during training is very important. Second, we develop a stacked architecture that includes warping of the second image with intermediate optical flow. Third, we elaborate on small displacements by introducing a sub-network specializing on small motions. FlowNet 2.0 is only marginally slower than the original FlowNet but decreases the estimation error by more than 50 methods, while running at interactive frame rates. Moreover, we present faster variants that allow optical flow computation at up to 140fps with accuracy matching the original FlowNet.

READ FULL TEXT

page 1

page 2

page 7

page 8

page 11

page 12

page 15

page 16

research
02/20/2018

Uncertainty Estimates for Optical Flow with Multi-Hypotheses Networks

Recent work has shown that optical flow estimation can be formulated as ...
research
02/25/2020

ScopeFlow: Dynamic Scene Scoping for Optical Flow

We propose to modify the common training protocols of optical flow, lead...
research
07/08/2022

Complementing Brightness Constancy with Deep Networks for Optical Flow Prediction

State-of-the-art methods for optical flow estimation rely on deep learni...
research
03/15/2023

Rethinking Optical Flow from Geometric Matching Consistent Perspective

Optical flow estimation is a challenging problem remaining unsolved. Rec...
research
04/11/2016

Beyond Brightness Constancy: Learning Noise Models for Optical Flow

Optical flow is typically estimated by minimizing a "data cost" and an o...
research
03/15/2019

A Lightweight Optical Flow CNN - Revisiting Data Fidelity and Regularization

Over four decades, the majority addresses the problem of optical flow es...
research
06/01/2017

TransFlow: Unsupervised Motion Flow by Joint Geometric and Pixel-level Estimation

We address unsupervised optical flow estimation for ego-centric motion. ...

Please sign up or login with your details

Forgot password? Click here to reset