Unsupervised Learning of Depth, Optical Flow and Pose with Occlusion from 3D Geometry

03/02/2020
by   Guangming Wang, et al.
9

In autonomous driving, monocular sequences contain lots of information. Monocular depth estimation, camera ego-motion estimation and optical flow estimation in consecutive frames are high-profile concerns recently. By analyzing tasks above, pixels in the first frame are modeled into three parts: the rigid region, the non-rigid region, and the occluded region. In joint unsupervised training of depth and pose, we can segment the occluded region explicitly. The occlusion information is used in unsupervised learning of depth, pose and optical flow, as the image reconstructed by depth, pose and flow will be invalid in occluded regions. A less-than-mean mask is designed to further exclude the mismatched pixels which are interfered with motion or illumination change in the training of depth and pose networks. This method is also used to exclude some trivial mismatched pixels in the training of the flow net. Maximum normalization is proposed for smoothness term of depth-pose networks to restrain degradation in textureless regions. In the occluded region, as depth and camera motion can provide more reliable motion estimation, they can be used to instruct unsupervised learning of flow. Our experiments in KITTI dataset demonstrate that the model based on three regions, full and explicit segmentation of occlusion, rigid region and non-rigid region with corresponding unsupervised losses can improve performance on three tasks significantly.

READ FULL TEXT

page 1

page 3

page 6

page 7

page 10

page 12

page 13

research
07/08/2021

NccFlow: Unsupervised Learning of Optical Flow With Non-occlusion from Geometry

Optical flow estimation is a fundamental problem of computer vision and ...
research
06/08/2022

Unsupervised Learning of 3D Scene Flow from Monocular Camera

Scene flow represents the motion of points in the 3D space, which is the...
research
08/25/2022

A Compacted Structure for Cross-domain learning on Monocular Depth and Flow Estimation

Accurate motion and depth recovery is important for many robot vision ta...
research
10/19/2010

3-D Rigid Models from Partial Views - Global Factorization

The so-called factorization methods recover 3-D rigid structure from mot...
research
12/01/2016

Unsupervised learning of image motion by recomposing sequences

We propose a new method for learning a representation of image motion in...
research
12/20/2018

Robustness Meets Deep Learning: An End-to-End Hybrid Pipeline for Unsupervised Learning of Egomotion

In this work, we propose a method that combines unsupervised deep learni...
research
03/06/2018

GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose

We propose GeoNet, a jointly unsupervised learning framework for monocul...

Please sign up or login with your details

Forgot password? Click here to reset