Ray Priors through Reprojection: Improving Neural Radiance Fields for Novel View Extrapolation

05/12/2022
by   Jian Zhang, et al.
0

Neural Radiance Fields (NeRF) have emerged as a potent paradigm for representing scenes and synthesizing photo-realistic images. A main limitation of conventional NeRFs is that they often fail to produce high-quality renderings under novel viewpoints that are significantly different from the training viewpoints. In this paper, instead of exploiting few-shot image synthesis, we study the novel view extrapolation setting that (1) the training images can well describe an object, and (2) there is a notable discrepancy between the training and test viewpoints' distributions. We present RapNeRF (RAy Priors) as a solution. Our insight is that the inherent appearances of a 3D surface's arbitrary visible projections should be consistent. We thus propose a random ray casting policy that allows training unseen views using seen views. Furthermore, we show that a ray atlas pre-computed from the observed rays' viewing directions could further enhance the rendering quality for extrapolated views. A main limitation is that RapNeRF would remove the strong view-dependent effects because it leverages the multi-view consistency property.

READ FULL TEXT

page 2

page 4

page 6

page 8

research
04/01/2021

Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis

We present DietNeRF, a 3D neural scene representation estimated from a f...
research
07/27/2023

MapNeRF: Incorporating Map Priors into Neural Radiance Fields for Driving View Simulation

Simulating camera sensors is a crucial task in autonomous driving. Altho...
research
12/04/2020

XraySyn: Realistic View Synthesis From a Single Radiograph Through CT Priors

A radiograph visualizes the internal anatomy of a patient through the us...
research
06/30/2023

MARF: The Medial Atom Ray Field Object Representation

We propose Medial Atom Ray Fields (MARFs), a novel neural object represe...
research
11/23/2022

PANeRF: Pseudo-view Augmentation for Improved Neural Radiance Fields Based on Few-shot Inputs

The method of neural radiance fields (NeRF) has been developed in recent...
research
06/30/2023

FlipNeRF: Flipped Reflection Rays for Few-shot Novel View Synthesis

Neural Radiance Field (NeRF) has been a mainstream in novel view synthes...
research
12/31/2021

InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Rendering

We present an information-theoretic regularization technique for few-sho...

Please sign up or login with your details

Forgot password? Click here to reset