Unsupervised Learning of Depth, Optical Flow and Pose with Occlusion from 3D Geometry

by   Guangming Wang, et al.

In autonomous driving, monocular sequences contain lots of information. Monocular depth estimation, camera ego-motion estimation and optical flow estimation in consecutive frames are high-profile concerns recently. By analyzing tasks above, pixels in the first frame are modeled into three parts: the rigid region, the non-rigid region, and the occluded region. In joint unsupervised training of depth and pose, we can segment the occluded region explicitly. The occlusion information is used in unsupervised learning of depth, pose and optical flow, as the image reconstructed by depth, pose and flow will be invalid in occluded regions. A less-than-mean mask is designed to further exclude the mismatched pixels which are interfered with motion or illumination change in the training of depth and pose networks. This method is also used to exclude some trivial mismatched pixels in the training of the flow net. Maximum normalization is proposed for smoothness term of depth-pose networks to restrain degradation in textureless regions. In the occluded region, as depth and camera motion can provide more reliable motion estimation, they can be used to instruct unsupervised learning of flow. Our experiments in KITTI dataset demonstrate that the model based on three regions, full and explicit segmentation of occlusion, rigid region and non-rigid region with corresponding unsupervised losses can improve performance on three tasks significantly.


page 1

page 3

page 6

page 7

page 10

page 12

page 13


NccFlow: Unsupervised Learning of Optical Flow With Non-occlusion from Geometry

Optical flow estimation is a fundamental problem of computer vision and ...

Unsupervised Learning of 3D Scene Flow from Monocular Camera

Scene flow represents the motion of points in the 3D space, which is the...

A Compacted Structure for Cross-domain learning on Monocular Depth and Flow Estimation

Accurate motion and depth recovery is important for many robot vision ta...

3-D Rigid Models from Partial Views - Global Factorization

The so-called factorization methods recover 3-D rigid structure from mot...

Unsupervised learning of image motion by recomposing sequences

We propose a new method for learning a representation of image motion in...

Robustness Meets Deep Learning: An End-to-End Hybrid Pipeline for Unsupervised Learning of Egomotion

In this work, we propose a method that combines unsupervised deep learni...

GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose

We propose GeoNet, a jointly unsupervised learning framework for monocul...

Please sign up or login with your details

Forgot password? Click here to reset