Joint Unsupervised Learning of Optical Flow and Egomotion with Bi-Level Optimization

02/26/2020
by   Shihao Jiang, et al.
3

We address the problem of joint optical flow and camera motion estimation in rigid scenes by incorporating geometric constraints into an unsupervised deep learning framework. Unlike existing approaches which rely on brightness constancy and local smoothness for optical flow estimation, we exploit the global relationship between optical flow and camera motion using epipolar geometry. In particular, we formulate the prediction of optical flow and camera motion as a bi-level optimization problem, consisting of an upper-level problem to estimate the flow that conforms to the predicted camera motion, and a lower-level problem to estimate the camera motion given the predicted optical flow. We use implicit differentiation to enable back-propagation through the lower-level geometric optimization layer independent of its implementation, allowing end-to-end training of the network. With globally-enforced geometric constraints, we are able to improve the quality of the estimated optical flow in challenging scenarios and obtain better camera motion estimates compared to other unsupervised learning methods.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 7

page 8

09/05/2018

DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency

We present an unsupervised learning framework for simultaneously trainin...
03/25/2021

GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning

Existing optical flow methods are erroneous in challenging scenes, such ...
06/01/2017

TransFlow: Unsupervised Motion Flow by Joint Geometric and Pixel-level Estimation

We address unsupervised optical flow estimation for ego-centric motion. ...
04/08/2019

Unsupervised Deep Epipolar Flow for Stationary or Dynamic Scenes

Unsupervised deep learning for optical flow computation has achieved pro...
05/09/2018

Layered Optical Flow Estimation Using a Deep Neural Network with a Soft Mask

Using a layered representation for motion estimation has the advantage o...
02/16/2016

Fast, Robust, Continuous Monocular Egomotion Computation

We propose robust methods for estimating camera egomotion in noisy, real...
09/23/2018

Unsupervised Learning of Dense Optical Flow and Depth from Sparse Event Data

In this work we present unsupervised learning of depth and motion from s...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.