Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image

09/21/2017
by   Fangchang Ma, et al.
0

We consider the problem of dense depth prediction from a sparse set of depth measurements and a single RGB image. Since depth estimation from monocular images alone is inherently ambiguous and unreliable, we introduce additional sparse depth samples, which are either collected from a low-resolution depth sensor or computed from SLAM, to attain a higher level of robustness and accuracy. We propose the use of a single regression network to learn directly from the RGB-D raw data, and explore the impact of number of depth samples on prediction accuracy. Our experiments show that, as compared to using only RGB images, the addition of 100 spatially random depth samples reduces the prediction root-mean-square error by half in the NYU-Depth-v2 indoor dataset. It also boosts the percentage of reliable prediction from 59 more challenging KITTI driving dataset. We demonstrate two applications of the proposed algorithm: serving as a plug-in module in SLAM to convert sparse maps to dense maps, and creating much denser point clouds from low-resolution LiDARs. Codes and video demonstration are publicly available.

READ FULL TEXT

page 1

page 5

page 6

page 7

research
04/08/2018

Estimating Depth from RGB and Sparse Sensing

We present a deep model that can accurately produce dense depth maps giv...
research
03/30/2021

Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completion

Estimating scene geometry from data obtained with cost-effective sensors...
research
12/10/2021

Sparse Depth Completion with Semantic Mesh Deformation Optimization

Sparse depth measurements are widely available in many applications such...
research
06/07/2020

DeepRelativeFusion: Dense Monocular SLAM using Single-Image Relative Depth Prediction

Traditional monocular visual simultaneous localization and mapping (SLAM...
research
04/01/2021

A Front-End for Dense Monocular SLAM using a Learned Outlier Mask Prior

Recent achievements in depth prediction from a single RGB image have pow...
research
12/02/2022

Sparse SPN: Depth Completion from Sparse Keypoints

Our long term goal is to use image-based depth completion to quickly cre...
research
08/20/2021

Deep Virtual Markers for Articulated 3D Shapes

We propose deep virtual markers, a framework for estimating dense and ac...

Please sign up or login with your details

Forgot password? Click here to reset