Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering

by   Vincent Sitzmann, et al.

Inferring representations of 3D scenes from 2D observations is a fundamental problem of computer graphics, computer vision, and artificial intelligence. Emerging 3D-structured neural scene representations are a promising approach to 3D scene understanding. In this work, we propose a novel neural scene representation, Light Field Networks or LFNs, which represent both geometry and appearance of the underlying 3D scene in a 360-degree, four-dimensional light field parameterized via a neural implicit representation. Rendering a ray from an LFN requires only a *single* network evaluation, as opposed to hundreds of evaluations per ray for ray-marching or volumetric based renderers in 3D-structured neural scene representations. In the setting of simple scenes, we leverage meta-learning to learn a prior over LFNs that enables multi-view consistent light field reconstruction from as little as a single image observation. This results in dramatic reductions in time and memory complexity, and enables real-time rendering. The cost of storing a 360-degree light field via an LFN is two orders of magnitude lower than conventional methods such as the Lumigraph. Utilizing the analytical differentiability of neural implicit representations and a novel parameterization of light space, we further demonstrate the extraction of sparse depth maps from LFNs.


page 5

page 7

page 16


Unsupervised Discovery and Composition of Object Light Fields

Neural scene representations, both continuous and discrete, have recentl...

Learning Neural Transmittance for Efficient Rendering of Reflectance Fields

Recently neural volumetric representations such as neural reflectance fi...

GRF: Learning a General Radiance Field for 3D Scene Representation and Rendering

We present a simple yet powerful implicit neural function that can repre...

DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks

The recent research explosion around implicit neural representations, su...

Neural Volumes: Learning Dynamic Renderable Volumes from Images

Modeling and rendering of dynamic scenes is challenging, as natural scen...

KiloNeuS: Implicit Neural Representations with Real-Time Global Illumination

The latest trends in inverse rendering techniques for reconstruction use...

Fristograms: Revealing and Exploiting Light Field Internals

In recent years, light field (LF) capture and processing has become an i...

Please sign up or login with your details

Forgot password? Click here to reset