Depth Completion with Twin Surface Extrapolation at Occlusion Boundaries

04/06/2021
by   Saif Imran, et al.
1

Depth completion starts from a sparse set of known depth values and estimates the unknown depths for the remaining image pixels. Most methods model this as depth interpolation and erroneously interpolate depth pixels into the empty space between spatially distinct objects, resulting in depth-smearing across occlusion boundaries. Here we propose a multi-hypothesis depth representation that explicitly models both foreground and background depths in the difficult occlusion-boundary regions. Our method can be thought of as performing twin-surface extrapolation, rather than interpolation, in these regions. Next our method fuses these extrapolated surfaces into a single depth image leveraging the image data. Key to our method is the use of an asymmetric loss function that operates on a novel twin-surface representation. This enables us to train a network to simultaneously do surface extrapolation and surface fusion. We characterize our loss function and compare with other common losses. Finally, we validate our method on three different datasets; KITTI, an outdoor real-world dataset, NYU2, indoor real-world depth dataset and Virtual KITTI, a photo-realistic synthetic dataset with dense groundtruth, and demonstrate improvement over the state of the art.

READ FULL TEXT

page 1

page 3

page 6

page 7

page 11

page 12

page 13

page 14

research
12/02/2018

DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image

In this paper, we propose a deep learning architecture that produces acc...
research
03/25/2018

Deep Depth Completion of a Single RGB-D Image

The goal of our work is to complete the depth channel of an RGB-D image....
research
07/01/2019

Pano Popups: Indoor 3D Reconstruction with a Plane-Aware Network

In this work we present a method to train a plane-aware convolutional ne...
research
03/13/2019

Depth Coefficients for Depth Completion

Depth completion involves estimating a dense depth image from sparse dep...
research
03/07/2022

Least Square Estimation Network for Depth Completion

Depth completion is a fundamental task in computer vision and robotics r...
research
03/30/2021

DynOcc: Learning Single-View Depth from Dynamic Occlusion Cues

Recently, significant progress has been made in single-view depth estima...
research
04/17/2021

A Surface Geometry Model for LiDAR Depth Completion

LiDAR depth completion is a task that predicts depth values for every pi...

Please sign up or login with your details

Forgot password? Click here to reset