DeepAI
Log In Sign Up

LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR

09/08/2021
by   Florent Bartoccioni, et al.
0

Vision-based depth estimation is a key feature in autonomous systems, which often relies on a single camera or several independent ones. In such a monocular setup, dense depth is obtained with either additional input from one or several expensive LiDARs, e.g., with 64 beams, or camera-only methods, which suffer from scale-ambiguity and infinite-depth problems. In this paper, we propose a new alternative of densely estimating metric depth by combining a monocular camera with a light-weight LiDAR, e.g., with 4 beams, typical of today's automotive-grade mass-produced laser scanners. Inspired by recent self-supervised methods, we introduce a novel framework, called LiDARTouch, to estimate dense depth maps from monocular images with the help of “touches” of LiDAR, i.e., without the need for dense ground-truth depth. In our setup, the minimal LiDAR input contributes on three different levels: as an additional model's input, in a self-supervised LiDAR reconstruction objective function, and to estimate changes of pose (a key component of self-supervised depth estimation architectures). Our LiDARTouch framework achieves new state of the art in self-supervised depth estimation on the KITTI dataset, thus supporting our choices of integrating the very sparse LiDAR signal with other visual features. Moreover, we show that the use of a few-beam LiDAR alleviates scale ambiguity and infinite-depth issues that camera-only methods suffer from. We also demonstrate that methods from the fully-supervised depth-completion literature can be adapted to a self-supervised regime with a minimal LiDAR signal.

READ FULL TEXT

page 3

page 5

page 8

page 10

page 12

page 13

09/30/2020

Monocular Differentiable Rendering for Self-Supervised 3D Object Detection

3D object detection from monocular images is an ill-posed problem due to...
04/12/2020

Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applications

In recent years, self-supervised methods for monocular depth estimation ...
12/16/2020

SimuGAN: Unsupervised forward modeling and optimal design of a LIDAR Camera

Energy-saving LIDAR camera for short distances estimates an object's dis...
09/16/2020

Calibrating Self-supervised Monocular Depth Estimation

In the recent years, many methods demonstrated the ability of neural net...
12/04/2021

Gated2Gated: Self-Supervised Depth Estimation from Gated Images

Gated cameras hold promise as an alternative to scanning LiDAR sensors w...
11/16/2022

Self-supervised Egomotion and Depth Learning via Bi-directional Coarse-to-Fine Scale Recovery

Self-supervised learning of egomotion and depth has recently attracted g...
03/11/2020

Uncertainty depth estimation with gated images for 3D reconstruction

Gated imaging is an emerging sensor technology for self-driving cars tha...