NeuralMVS: Bridging Multi-View Stereo and Novel View Synthesis

08/09/2021
by   Radu Alexandru Rosu, et al.
7

Multi-View Stereo (MVS) is a core task in 3D computer vision. With the surge of novel deep learning methods, learned MVS has surpassed the accuracy of classical approaches, but still relies on building a memory intensive dense cost volume. Novel View Synthesis (NVS) is a parallel line of research and has recently seen an increase in popularity with Neural Radiance Field (NeRF) models, which optimize a per scene radiance field. However, NeRF methods do not generalize to novel scenes and are slow to train and test. We propose to bridge the gap between these two methodologies with a novel network that can recover 3D scene geometry as a distance function, together with high-resolution color images. Our method uses only a sparse set of images as input and can generalize well to novel scenes. Additionally, we propose a coarse-to-fine sphere tracing approach in order to significantly increase speed. We show on various datasets that our method reaches comparable accuracy to per-scene optimized methods while being able to generalize and running significantly faster.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 5

04/14/2021

Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenes

Recent neural view synthesis methods have achieved impressive quality an...
02/25/2021

IBRNet: Learning Multi-View Image-Based Rendering

We present a method that synthesizes novel views of complex scenes by in...
09/06/2021

Point-Based Neural Rendering with Per-View Optimization

There has recently been great interest in neural rendering methods. Some...
04/06/2019

360 Panorama Synthesis from a Sparse Set of Images with Unknown FOV

360 images represent scenes captured in all possible viewing directions....
04/25/2018

Fast View Synthesis with Deep Stereo Vision

Novel view synthesis is an important problem in computer vision and grap...
12/02/2020

PatchmatchNet: Learned Multi-View Patchmatch Stereo

We present PatchmatchNet, a novel and learnable cascade formulation of P...
02/19/2021

Compact and adaptive multiplane images for view synthesis

Recently, learning methods have been designed to create Multiplane Image...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.