ProFlow: Learning to Predict Optical Flow

06/03/2018
by   Daniel Maurer, et al.
0

Temporal coherence is a valuable source of information in the context of optical flow estimation. However, finding a suitable motion model to leverage this information is a non-trivial task. In this paper we propose an unsupervised online learning approach based on a convolutional neural network (CNN) that estimates such a motion model individually for each frame. By relating forward and backward motion these learned models not only allow to infer valuable motion information based on the backward flow, they also help to improve the performance at occlusions, where a reliable prediction is particularly useful. Moreover, our learned models are spatially variant and hence allow to estimate non-rigid motion per construction. This, in turns, allows to overcome the major limitation of recent rigidity-based approaches that seek to improve the estimation by incorporating additional stereo/SfM constraints. Experiments demonstrate the usefulness of our new approach. They not only show a consistent improvement of up to 27 (KITTI 2012, KITTI 2015, MPI Sintel) compared to a baseline without prediction, they also show top results for the MPI Sintel benchmark -- the one of the three benchmarks that contains the largest amount of non-rigid motion.

READ FULL TEXT

page 3

page 5

page 8

page 13

page 14

page 15

page 16

research
05/03/2017

Optical Flow in Mostly Rigid Scenes

The optical flow of natural scenes is a combination of the motion of the...
research
04/13/2021

MESD: Exploring Optical Flow Assessment on Edge of Motion Objects with Motion Edge Structure Difference

The optical flow estimation has been assessed in various applications. I...
research
07/31/2023

SAMFlow: Eliminating Any Fragmentation in Optical Flow with Segment Anything Model

Optical flow estimation aims to find the 2D dense motion field between t...
research
04/08/2021

ASFlow: Unsupervised Optical Flow Learning with Adaptive Pyramid Sampling

We present an unsupervised optical flow estimation method by proposing a...
research
07/12/2022

M-FUSE: Multi-frame Fusion for Scene Flow Estimation

Recently, neural network for scene flow estimation show impressive resul...
research
04/20/2020

How Do Neural Networks Estimate Optical Flow? A Neuropsychology-Inspired Study

End-to-end trained convolutional neural networks have led to a breakthro...
research
11/16/2018

Topology-Aware Non-Rigid Point Cloud Registration

In this paper, we introduce a non-rigid registration pipeline for pairs ...

Please sign up or login with your details

Forgot password? Click here to reset