Neural Reflectance Fields for Appearance Acquisition

by   Sai Bi, et al.
berkeley college
University of California, San Diego

We present Neural Reflectance Fields, a novel deep scene representation that encodes volume density, normal and reflectance properties at any 3D point in a scene using a fully-connected neural network. We combine this representation with a physically-based differentiable ray marching framework that can render images from a neural reflectance field under any viewpoint and light. We demonstrate that neural reflectance fields can be estimated from images captured with a simple collocated camera-light setup, and accurately model the appearance of real-world scenes with complex geometry and reflectance. Once estimated, they can be used to render photo-realistic images under novel viewpoint and (non-collocated) lighting conditions and accurately reproduce challenging effects like specularities, shadows and occlusions. This allows us to perform high-quality view synthesis and relighting that is significantly better than previous methods. We also demonstrate that we can compose the estimated neural reflectance field of a real scene with traditional scene models and render them using standard Monte Carlo rendering engines. Our work thus enables a complete pipeline from high-quality and practical appearance acquisition to 3D scene composition and rendering.


page 1

page 2

page 3

page 4

page 5

page 6

page 7

page 11


Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images

We present a deep learning approach to reconstruct scene appearance from...

Photon-Driven Neural Path Guiding

Although Monte Carlo path tracing is a simple and effective algorithm to...

SC-NeRF: Self-Correcting Neural Radiance Field with Sparse Views

In recent studies, the generalization of neural radiance fields for nove...

Deep Appearance Prefiltering

Physically based rendering of complex scenes can be prohibitively costly...

Learning Neural Transmittance for Efficient Rendering of Reflectance Fields

Recently neural volumetric representations such as neural reflectance fi...

Progressively-connected Light Field Network for Efficient View Synthesis

This paper presents a Progressively-connected Light Field network (ProLi...

Neural Projection Mapping Using Reflectance Fields

We introduce a high resolution spatially adaptive light source, or a pro...

Please sign up or login with your details

Forgot password? Click here to reset