MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo

03/29/2021
by   Anpei Chen, et al.
0

We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis. Unlike prior works on neural radiance fields that consider per-scene optimization on densely captured images, we propose a generic deep neural network that can reconstruct radiance fields from only three nearby input views via fast network inference. Our approach leverages plane-swept cost volumes (widely used in multi-view stereo) for geometry-aware scene reasoning, and combines this with physically based volume rendering for neural radiance field reconstruction. We train our network on real objects in the DTU dataset, and test it on three different datasets to evaluate its effectiveness and generalizability. Our approach can generalize across scenes (even indoor scenes, completely different from our training scenes of objects) and generate realistic view synthesis results using only three input images, significantly outperforming concurrent works on generalizable radiance field reconstruction. Moreover, if dense images are captured, our estimated radiance field representation can be easily fine-tuned; this leads to fast per-scene reconstruction with higher rendering quality and substantially less optimization time than NeRF.

READ FULL TEXT

page 1

page 6

page 8

research
04/14/2021

Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenes

Recent neural view synthesis methods have achieved impressive quality an...
research
04/17/2023

Learning to Render Novel Views from Wide-Baseline Stereo Pairs

We introduce a method for novel view synthesis given only a single wide-...
research
09/20/2023

Language-driven Object Fusion into Neural Radiance Fields with Pose-Conditioned Dataset Updates

Neural radiance field is an emerging rendering method that generates hig...
research
06/24/2021

Free-viewpoint Indoor Neural Relighting from Multi-view Stereo

We introduce a neural relighting algorithm for captured indoors scenes, ...
research
06/09/2023

GANeRF: Leveraging Discriminators to Optimize Neural Radiance Fields

Neural Radiance Fields (NeRF) have shown impressive novel view synthesis...
research
07/27/2023

Improved Neural Radiance Fields Using Pseudo-depth and Fusion

Since the advent of Neural Radiance Fields, novel view synthesis has rec...
research
07/04/2022

Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentations

Neural Radiance Field (NeRF) regresses a neural parameterized scene by d...

Please sign up or login with your details

Forgot password? Click here to reset