DeepAI
Log In Sign Up

Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering

06/04/2021
by   Vincent Sitzmann, et al.
14

Inferring representations of 3D scenes from 2D observations is a fundamental problem of computer graphics, computer vision, and artificial intelligence. Emerging 3D-structured neural scene representations are a promising approach to 3D scene understanding. In this work, we propose a novel neural scene representation, Light Field Networks or LFNs, which represent both geometry and appearance of the underlying 3D scene in a 360-degree, four-dimensional light field parameterized via a neural implicit representation. Rendering a ray from an LFN requires only a *single* network evaluation, as opposed to hundreds of evaluations per ray for ray-marching or volumetric based renderers in 3D-structured neural scene representations. In the setting of simple scenes, we leverage meta-learning to learn a prior over LFNs that enables multi-view consistent light field reconstruction from as little as a single image observation. This results in dramatic reductions in time and memory complexity, and enables real-time rendering. The cost of storing a 360-degree light field via an LFN is two orders of magnitude lower than conventional methods such as the Lumigraph. Utilizing the analytical differentiability of neural implicit representations and a novel parameterization of light space, we further demonstrate the extraction of sparse depth maps from LFNs.

READ FULL TEXT

page 5

page 7

page 16

05/08/2022

Unsupervised Discovery and Composition of Object Light Fields

Neural scene representations, both continuous and discrete, have recentl...
10/09/2020

GRF: Learning a General Radiance Field for 3D Scene Representation and Rendering

We present a simple yet powerful implicit neural function that can repre...
10/25/2021

Learning Neural Transmittance for Efficient Rendering of Reflectance Fields

Recently neural volumetric representations such as neural reflectance fi...
03/04/2021

DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks

The recent research explosion around implicit neural representations, su...
06/18/2019

Neural Volumes: Learning Dynamic Renderable Volumes from Images

Modeling and rendering of dynamic scenes is challenging, as natural scen...
06/22/2022

KiloNeuS: Implicit Neural Representations with Real-Time Global Illumination

The latest trends in inverse rendering techniques for reconstruction use...
07/22/2021

Fristograms: Revealing and Exploiting Light Field Internals

In recent years, light field (LF) capture and processing has become an i...