3DVNet: Multi-View Depth Prediction and Volumetric Refinement

12/01/2021
by   Alexander Rich, et al.
0

We present 3DVNet, a novel multi-view stereo (MVS) depth-prediction method that combines the advantages of previous depth-based and volumetric MVS approaches. Our key idea is the use of a 3D scene-modeling network that iteratively updates a set of coarse depth predictions, resulting in highly accurate predictions which agree on the underlying scene geometry. Unlike existing depth-prediction techniques, our method uses a volumetric 3D convolutional neural network (CNN) that operates in world space on all depth maps jointly. The network can therefore learn meaningful scene-level priors. Furthermore, unlike existing volumetric MVS techniques, our 3D CNN operates on a feature-augmented point cloud, allowing for effective aggregation of multi-view information and flexible iterative refinement of depth maps. Experimental results show our method exceeds state-of-the-art accuracy in both depth prediction and 3D reconstruction metrics on the ScanNet dataset, as well as a selection of scenes from the TUM-RGBD and ICL-NUIM datasets. This shows that our method is both effective and generalizes to new settings.

READ FULL TEXT

page 4

page 8

research
08/12/2019

Point-Based Multi-View Stereo Network

We introduce Point-MVSNet, a novel point-based deep framework for multi-...
research
09/06/2021

Point-Based Neural Rendering with Per-View Optimization

There has recently been great interest in neural rendering methods. Some...
research
06/27/2018

Learn-to-Score: Efficient 3D Scene Exploration by Predicting View Utility

Camera equipped drones are nowadays being used to explore large scenes a...
research
05/21/2019

Mesh-based Camera Pairs Selection and Occlusion-Aware Masking for Mesh Refinement

Many Multi-View-Stereo algorithms extract a 3D mesh model of a scene, af...
research
04/03/2019

StereoDRNet: Dilated Residual Stereo Net

We propose a system that uses a convolution neural network (CNN) to esti...
research
11/19/2020

Deep Multi-view Depth Estimation with Predicted Uncertainty

In this paper, we address the problem of estimating dense depth from a s...
research
04/12/2016

Volumetric and Multi-View CNNs for Object Classification on 3D Data

3D shape models are becoming widely available and easier to capture, mak...

Please sign up or login with your details

Forgot password? Click here to reset