DeepAI
Log In Sign Up

Neural Sparse Voxel Fields

07/22/2020
by   Lingjie Liu, et al.
0

Photo-realistic free-viewpoint rendering of real-world scenes using classical computer graphics techniques is challenging, because it requires the difficult step of capturing detailed appearance and geometry models. Recent studies have demonstrated promising results by learning scene representations that implicitly encode both geometry and appearance without 3D supervision. However, existing approaches in practice often show blurry renderings caused by the limited network capacity or the difficulty in finding accurate intersections of camera rays with the scene geometry. Synthesizing high-resolution imagery from these representations often requires time-consuming optical ray marching. In this work, we introduce Neural Sparse Voxel Fields (NSVF), a new neural scene representation for fast and high-quality free-viewpoint rendering. NSVF defines a set of voxel-bounded implicit fields organized in a sparse voxel octree to model local properties in each cell. We progressively learn the underlying voxel structures with a diffentiable ray-marching operation from only a set of posed RGB images. With the sparse voxel octree structure, rendering novel views can be accelerated by skipping the voxels containing no relevant scene content. Our method is over 10 times faster than the state-of-the-art (namely, NeRF) at inference time while achieving higher quality results. Furthermore, by utilizing an explicit sparse voxel representation, our method can easily be applied to scene editing and scene composition. We also demonstrate several challenging tasks, including multi-scene learning, free-viewpoint rendering of a moving human, and large-scale scene rendering.

READ FULL TEXT

page 6

page 7

page 8

page 9

page 17

page 18

page 20

06/15/2022

VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids

State-of-the-art 3D-aware generative models rely on coordinate-based MLP...
08/17/2022

PDRF: Progressively Deblurring Radiance Field for Fast and Robust Scene Reconstruction from Blurry Images

We present Progressively Deblurring Radiance Field (PDRF), a novel appro...
08/21/2022

Vox-Surf: Voxel-based Implicit Surface Representation

Virtual content creation and interaction play an important role in moder...
12/10/2019

Neural Voxel Renderer: Learning an Accurate and Controllable Rendering Tool

We present a neural rendering framework that maps a voxelized scene into...
11/07/2022

QRF: Implicit Neural Representations with Quantum Radiance Fields

Photorealistic rendering of real-world scenes is a tremendous challenge ...
11/19/2021

DIVeR: Real-time and Accurate Neural Radiance Fields with Deterministic Integration for Volume Rendering

DIVeR builds on the key ideas of NeRF and its variants – density models ...