PlaNeRF: SVD Unsupervised 3D Plane Regularization for NeRF Large-Scale Scene Reconstruction

05/26/2023
by   Fusang Wang, et al.
0

Neural Radiance Fields (NeRF) enable 3D scene reconstruction from 2D images and camera poses for Novel View Synthesis (NVS). Although NeRF can produce photorealistic results, it often suffers from overfitting to training views, leading to poor geometry reconstruction, especially in low-texture areas. This limitation restricts many important applications which require accurate geometry, such as extrapolated NVS, HD mapping and scene editing. To address this limitation, we propose a new method to improve NeRF's 3D structure using only RGB images and semantic maps. Our approach introduces a novel plane regularization based on Singular Value Decomposition (SVD), that does not rely on any geometric prior. In addition, we leverage the Structural Similarity Index Measure (SSIM) in our loss design to properly initialize the volumetric representation of NeRF. Quantitative and qualitative results show that our method outperforms popular regularization approaches in accurate geometry reconstruction for large-scale outdoor scenes and achieves SoTA rendering quality on the KITTI-360 NVS benchmark.

READ FULL TEXT

page 1

page 6

page 7

page 8

page 9

page 10

research
07/21/2023

RICo: Rotate-Inpaint-Complete for Generalizable Scene Reconstruction

General scene reconstruction refers to the task of estimating the full 3...
research
03/03/2023

Multi-Plane Neural Radiance Fields for Novel View Synthesis

Novel view synthesis is a long-standing problem that revolves around ren...
research
03/26/2023

Clean-NeRF: Reformulating NeRF to account for View-Dependent Observations

While Neural Radiance Fields (NeRFs) had achieved unprecedented novel vi...
research
03/30/2023

Enhanced Stable View Synthesis

We introduce an approach to enhance the novel view synthesis from images...
research
02/20/2018

Novel View Synthesis for Large-scale Scene using Adversarial Loss

Novel view synthesis aims to synthesize new images from different viewpo...
research
07/07/2023

RGB-D Mapping and Tracking in a Plenoxel Radiance Field

Building on the success of Neural Radiance Fields (NeRFs), recent years ...
research
09/23/2022

PNeRF: Probabilistic Neural Scene Representations for Uncertain 3D Visual Mapping

Recently neural scene representations have provided very impressive resu...

Please sign up or login with your details

Forgot password? Click here to reset