DeepAI AI Chat
Log In Sign Up

Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video

by   Jia-Wang Bian, et al.

Recent work has shown that CNN-based depth and ego-motion estimators can be learned using unlabelled monocular videos. However, the performance is limited by unidentified moving objects that violate the underlying static scene assumption in geometric image reconstruction. More significantly, due to lack of proper constraints, networks output scale-inconsistent results over different samples, i.e., the ego-motion network cannot provide full camera trajectories over a long video sequence because of the per-frame scale ambiguity. This paper tackles these challenges by proposing a geometry consistency loss for scale-consistent predictions, and an induced self-discovered mask for handling moving objects and occlusions. Since we do not leverage multi-task learning like recent works, our framework is much simpler and more efficient. Extensive evaluation results demonstrate that our depth estimator achieves the state-of-the-art performance on the standard KITTI and Make3D datasets. Moreover, we show that our ego-motion network is able to predict a globally scale-consistent camera trajectory for long video sequences, and the resulting visual odometry accuracy is competitive with the state-of-the-art model that is trained using stereo videos. To the best of our knowledge, this is the first work to show that deep networks trained using monocular video snippets can predict globally scale-consistent camera trajectories over a long video sequence.


page 5

page 12


Unsupervised Scale-consistent Depth Learning from Video

We propose a monocular depth estimator SC-Depth, which requires only unl...

Sparse Representations for Object and Ego-motion Estimation in Dynamic Scenes

Dynamic scenes that contain both object motion and egomotion are a chall...

Masked GANs for Unsupervised Depth and Pose Prediction with Scale Consistency

Previous works have shown that adversarial learning can be used for unsu...

Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras

We present a novel method for simultaneous learning of depth, egomotion,...

Simple means Faster: Real-Time Human Motion Forecasting in Monocular First Person Videos on CPU

We present a simple, fast, and light-weight RNN based framework for fore...

Unsupervised Monocular Depth Learning with Integrated Intrinsics and Spatio-Temporal Constraints

Monocular depth inference has gained tremendous attention from researche...