NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis

12/07/2020
by   Pratul P. Srinivasan, et al.
0

We present a method that takes as input a set of images of a scene illuminated by unconstrained known lighting, and produces as output a 3D representation that can be rendered from novel viewpoints under arbitrary lighting conditions. Our method represents the scene as a continuous volumetric function parameterized as MLPs whose inputs are a 3D location and whose outputs are the following scene properties at that input location: volume density, surface normal, material parameters, distance to the first surface intersection in any direction, and visibility of the external environment in any direction. Together, these allow us to render novel views of the object under arbitrary lighting, including indirect illumination effects. The predicted visibility and surface intersection fields are critical to our model's ability to simulate direct and indirect illumination during training, because the brute-force techniques used by prior work are intractable for lighting conditions outside of controlled setups with a single light. Our method outperforms alternative approaches for recovering relightable 3D scene representations, and performs well in complex lighting settings that have posed a significant challenge to prior work.

READ FULL TEXT

page 1

page 2

page 3

page 5

page 6

page 9

page 10

page 11

research
05/29/2023

Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects

We develop a method that recovers the surface, materials, and illuminati...
research
01/26/2023

Screen Space Indirect Lighting with Visibility Bitmask

Horizon-based indirect illumination efficiently estimates a diffuse ligh...
research
06/03/2021

NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination

We address the problem of recovering the shape and spatially-varying ref...
research
08/09/2020

Neural Light Transport for Relighting and View Synthesis

The light transport (LT) of a scene describes how it appears under diffe...
research
04/24/2023

TensoIR: Tensorial Inverse Rendering

We propose TensoIR, a novel inverse rendering approach based on tensor f...
research
04/18/2023

Deep Dynamic Cloud Lighting

Sky illumination is a core source of lighting in rendering, and a substa...
research
04/20/2021

Shadow Neural Radiance Fields for Multi-view Satellite Photogrammetry

We present a new generic method for shadow-aware multi-view satellite ph...

Please sign up or login with your details

Forgot password? Click here to reset