Multi-view Monocular Depth and Uncertainty Prediction with Deep SfM in Dynamic Environments

01/21/2022
by   Christian Homeyer, et al.
0

3D reconstruction of depth and motion from monocular video in dynamic environments is a highly ill-posed problem due to scale ambiguities when projecting to the 2D image domain. In this work, we investigate the performance of the current State-of-the-Art (SotA) deep multi-view systems in such environments. We find that current supervised methods work surprisingly well despite not modelling individual object motions, but make systematic errors due to a lack of dense ground truth data. To detect such errors during usage, we extend the cost volume based Deep Video to Depth (DeepV2D) framework <cit.> with a learned uncertainty. Our Deep Video to certain Depth (DeepV2cD) model allows i) to perform en par or better with current SotA and ii) achieve a better uncertainty measure than the naive Shannon entropy. Our experiments show that a simple filter strategy based on the uncertainty can significantly reduce systematic errors. This results in cleaner reconstructions both on static and dynamic parts of the scene.

READ FULL TEXT

page 2

page 8

page 13

research
02/09/2017

Semi-Supervised Deep Learning for Monocular Depth Map Prediction

Supervised deep learning often suffers from the lack of sufficient train...
research
06/17/2023

Multi-view 3D Object Reconstruction and Uncertainty Modelling with Neural Shape Prior

3D object reconstruction is important for semantic scene understanding. ...
research
11/24/2022

SfM-TTR: Using Structure from Motion for Test-Time Refinement of Single-View Depth Networks

Estimating a dense depth map from a single view is geometrically ill-pos...
research
11/24/2020

MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera

In this paper, we propose MonoRec, a semi-supervised monocular dense rec...
research
12/28/2022

NeMo: 3D Neural Motion Fields from Multiple Video Instances of the Same Action

The task of reconstructing 3D human motion has wideranging applications....
research
04/18/2023

Learning to Fuse Monocular and Multi-view Cues for Multi-frame Depth Estimation in Dynamic Scenes

Multi-frame depth estimation generally achieves high accuracy relying on...
research
09/06/2019

Self-supervised Dense 3D Reconstruction from Monocular Endoscopic Video

We present a self-supervised learning-based pipeline for dense 3D recons...

Please sign up or login with your details

Forgot password? Click here to reset