Neural Sparse Voxel Fields

by   Lingjie Liu, et al.

Photo-realistic free-viewpoint rendering of real-world scenes using classical computer graphics techniques is challenging, because it requires the difficult step of capturing detailed appearance and geometry models. Recent studies have demonstrated promising results by learning scene representations that implicitly encode both geometry and appearance without 3D supervision. However, existing approaches in practice often show blurry renderings caused by the limited network capacity or the difficulty in finding accurate intersections of camera rays with the scene geometry. Synthesizing high-resolution imagery from these representations often requires time-consuming optical ray marching. In this work, we introduce Neural Sparse Voxel Fields (NSVF), a new neural scene representation for fast and high-quality free-viewpoint rendering. NSVF defines a set of voxel-bounded implicit fields organized in a sparse voxel octree to model local properties in each cell. We progressively learn the underlying voxel structures with a diffentiable ray-marching operation from only a set of posed RGB images. With the sparse voxel octree structure, rendering novel views can be accelerated by skipping the voxels containing no relevant scene content. Our method is over 10 times faster than the state-of-the-art (namely, NeRF) at inference time while achieving higher quality results. Furthermore, by utilizing an explicit sparse voxel representation, our method can easily be applied to scene editing and scene composition. We also demonstrate several challenging tasks, including multi-scene learning, free-viewpoint rendering of a moving human, and large-scale scene rendering.


page 6

page 7

page 8

page 9

page 17

page 18

page 20


VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids

State-of-the-art 3D-aware generative models rely on coordinate-based MLP...

PDRF: Progressively Deblurring Radiance Field for Fast and Robust Scene Reconstruction from Blurry Images

We present Progressively Deblurring Radiance Field (PDRF), a novel appro...

Vox-Surf: Voxel-based Implicit Surface Representation

Virtual content creation and interaction play an important role in moder...

Neural Voxel Renderer: Learning an Accurate and Controllable Rendering Tool

We present a neural rendering framework that maps a voxelized scene into...

QRF: Implicit Neural Representations with Quantum Radiance Fields

Photorealistic rendering of real-world scenes is a tremendous challenge ...

Tetra-NeRF: Representing Neural Radiance Fields Using Tetrahedra

Neural Radiance Fields (NeRFs) are a very recent and very popular approa...

DIVeR: Real-time and Accurate Neural Radiance Fields with Deterministic Integration for Volume Rendering

DIVeR builds on the key ideas of NeRF and its variants – density models ...

Please sign up or login with your details

Forgot password? Click here to reset