DeepAI AI Chat
Log In Sign Up

Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes

by   Zhengqi Li, et al.

We present a method to perform novel view and time synthesis of dynamic scenes, requiring only a monocular video with known camera poses as input. To do this, we introduce Neural Scene Flow Fields, a new representation that models the dynamic scene as a time-variant continuous function of appearance, geometry, and 3D scene motion. Our representation is optimized through a neural network to fit the observed input views. We show that our representation can be used for complex dynamic scenes, including thin structures, view-dependent effects, and natural degrees of motion. We conduct a number of experiments that demonstrate our approach significantly outperforms recent monocular view synthesis methods, and show qualitative results of space-time view synthesis on a variety of real-world videos.


page 1

page 4

page 5

page 6

page 7

page 8


Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video

We present Non-Rigid Neural Radiance Fields (NR-NeRF), a reconstruction ...

Dynamic View Synthesis from Dynamic Monocular Video

We present an algorithm for generating novel views at arbitrary viewpoin...

Detachable Novel Views Synthesis of Dynamic Scenes Using Distribution-Driven Neural Radiance Fields

Representing and synthesizing novel views in real-world dynamic scenes f...

Neural Radiance Flow for 4D View Synthesis and Video Processing

We present a method, Neural Radiance Flow (NeRFlow),to learn a 4D spatia...

Dynamic Scene Novel View Synthesis via Deferred Spatio-temporal Consistency

Structure from motion (SfM) enables us to reconstruct a scene via casual...

Free View Synthesis

We present a method for novel view synthesis from input images that are ...

wildNeRF: Complete view synthesis of in-the-wild dynamic scenes captured using sparse monocular data

We present a novel neural radiance model that is trainable in a self-sup...