NVS-MonoDepth: Improving Monocular Depth Prediction with Novel View Synthesis

12/22/2021
by   Zuria Bauer, et al.
10

Building upon the recent progress in novel view synthesis, we propose its application to improve monocular depth estimation. In particular, we propose a novel training method split in three main steps. First, the prediction results of a monocular depth network are warped to an additional view point. Second, we apply an additional image synthesis network, which corrects and improves the quality of the warped RGB image. The output of this network is required to look as similar as possible to the ground-truth view by minimizing the pixel-wise RGB reconstruction error. Third, we reapply the same monocular depth estimation onto the synthesized second view point and ensure that the depth predictions are consistent with the associated ground truth depth. Experimental results prove that our method achieves state-of-the-art or comparable performance on the KITTI and NYU-Depth-v2 datasets with a lightweight and simple vanilla U-Net architecture.

READ FULL TEXT

page 7

page 8

page 16

page 17

page 18

page 19

page 20

page 21

research
03/21/2018

Monocular Depth Estimation by Learning from Heterogeneous Datasets

Depth estimation provides essential information to perform autonomous dr...
research
05/21/2019

SharpNet: Fast and Accurate Recovery of Occluding Contours in Monocular Depth Estimation

We introduce SharpNet, a method that predicts an accurate depth map for ...
research
06/24/2021

FaDIV-Syn: Fast Depth-Independent View Synthesis

We introduce FaDIV-Syn, a fast depth-independent view synthesis method. ...
research
05/30/2023

DäRF: Boosting Radiance Fields from Sparse Inputs with Monocular Depth Adaptation

Neural radiance fields (NeRF) shows powerful performance in novel view s...
research
01/16/2019

Lightweight Markerless Monocular Face Capture with 3D Spatial Priors

We present a simple lightweight markerless facial performance capture fr...
research
08/03/2022

Gradient-based Uncertainty for Monocular Depth Estimation

In monocular depth estimation, disturbances in the image context, like m...
research
03/14/2020

Fast Depth Estimation for View Synthesis

Disparity/depth estimation from sequences of stereo images is an importa...

Please sign up or login with your details

Forgot password? Click here to reset