HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields

06/24/2021
by   Keunhong Park, et al.
23

Neural Radiance Fields (NeRF) are able to reconstruct scenes with unprecedented fidelity, and various recent works have extended NeRF to handle dynamic scenes. A common approach to reconstruct such non-rigid scenes is through the use of a learned deformation field mapping from coordinates in each input image into a canonical template coordinate space. However, these deformation-based approaches struggle to model changes in topology, as topological changes require a discontinuity in the deformation field, but these deformation fields are necessarily continuous. We address this limitation by lifting NeRFs into a higher dimensional space, and by representing the 5D radiance field corresponding to each individual input image as a slice through this "hyper-space". Our method is inspired by level set methods, which model the evolution of surfaces as slices through a higher dimensional surface. We evaluate our method on two tasks: (i) interpolating smoothly between "moments", i.e., configurations of the scene, seen in the input images while maintaining visual plausibility, and (ii) novel-view synthesis at fixed moments. We show that our method, which we dub HyperNeRF, outperforms existing methods on both tasks. Compared to Nerfies, HyperNeRF reduces average error rates by 4.1 interpolation and 8.6 Additional videos, results, and visualizations are available at https://hypernerf.github.io.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 6

page 7

page 8

page 10

page 11

page 13

page 14

05/06/2021

Animatable Neural Radiance Fields for Human Body Modeling

This paper addresses the challenge of reconstructing an animatable human...
11/25/2020

Deformable Neural Radiance Fields

We present the first method capable of photorealistically reconstructing...
11/27/2020

D-NeRF: Neural Radiance Fields for Dynamic Scenes

Neural rendering techniques combining machine learning with geometric re...
05/12/2021

Neural Trajectory Fields for Dynamic Novel View Synthesis

Recent approaches to render photorealistic views from a limited set of p...
08/20/2019

DefSLAM: Tracking and Mapping of Deforming Scenes from Monocular Sequences

We present the first monocular SLAM capable of operating in deforming sc...
12/11/2020

Learning Edge-Preserved Image Stitching from Large-Baseline Deep Homography

Image stitching is a classical and crucial technique in computer vision,...
11/22/2021

Neural Fields in Visual Computing and Beyond

Recent advances in machine learning have created increasing interest in ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.