Joint Unsupervised Learning of Optical Flow and Egomotion with Bi-Level Optimization

by   Shihao Jiang, et al.

We address the problem of joint optical flow and camera motion estimation in rigid scenes by incorporating geometric constraints into an unsupervised deep learning framework. Unlike existing approaches which rely on brightness constancy and local smoothness for optical flow estimation, we exploit the global relationship between optical flow and camera motion using epipolar geometry. In particular, we formulate the prediction of optical flow and camera motion as a bi-level optimization problem, consisting of an upper-level problem to estimate the flow that conforms to the predicted camera motion, and a lower-level problem to estimate the camera motion given the predicted optical flow. We use implicit differentiation to enable back-propagation through the lower-level geometric optimization layer independent of its implementation, allowing end-to-end training of the network. With globally-enforced geometric constraints, we are able to improve the quality of the estimated optical flow in challenging scenarios and obtain better camera motion estimates compared to other unsupervised learning methods.



There are no comments yet.


page 7

page 8


DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency

We present an unsupervised learning framework for simultaneously trainin...

GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning

Existing optical flow methods are erroneous in challenging scenes, such ...

TransFlow: Unsupervised Motion Flow by Joint Geometric and Pixel-level Estimation

We address unsupervised optical flow estimation for ego-centric motion. ...

Unsupervised Deep Epipolar Flow for Stationary or Dynamic Scenes

Unsupervised deep learning for optical flow computation has achieved pro...

Layered Optical Flow Estimation Using a Deep Neural Network with a Soft Mask

Using a layered representation for motion estimation has the advantage o...

Fast, Robust, Continuous Monocular Egomotion Computation

We propose robust methods for estimating camera egomotion in noisy, real...

Unsupervised Learning of Dense Optical Flow and Depth from Sparse Event Data

In this work we present unsupervised learning of depth and motion from s...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.