TöRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesis

09/30/2021
by   Benjamin Attal, et al.
4

Neural networks can represent and accurately reconstruct radiance fields for static 3D scenes (e.g., NeRF). Several works extend these to dynamic scenes captured with monocular video, with promising performance. However, the monocular setting is known to be an under-constrained problem, and so methods rely on data-driven priors for reconstructing dynamic content. We replace these priors with measurements from a time-of-flight (ToF) camera, and introduce a neural representation based on an image formation model for continuous-wave ToF cameras. Instead of working with processed depth maps, we model the raw ToF sensor measurements to improve reconstruction quality and avoid issues with low reflectance regions, multi-path interference, and a sensor's limited unambiguous depth range. We show that this approach improves robustness of dynamic scene reconstruction to erroneous calibration and large motions, and discuss the benefits and limitations of integrating RGB+ToF sensors that are now available on modern smartphones.

READ FULL TEXT

page 2

page 6

page 8

page 9

page 10

page 11

page 17

research
01/31/2014

Cross-calibration of Time-of-flight and Colour Cameras

Time-of-flight cameras provide depth information, which is complementary...
research
02/02/2020

Depth Map Estimation of Dynamic Scenes Using Prior Depth Information

Depth information is useful for many applications. Active depth sensors ...
research
11/26/2021

NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images

Neural Radiance Fields (NeRF) is a technique for high quality novel view...
research
06/30/2022

Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera

We propose Neural-DynamicReconstruction (NDR), a template-free method to...
research
09/12/2018

Multi range Real-time depth inference from a monocular stabilized footage using a Fully Convolutional Neural Network

Using a neural network architecture for depth map inference from monocul...
research
02/27/2023

BaLi-RF: Bandlimited Radiance Fields for Dynamic Scene Modeling

Reasoning the 3D structure of a non-rigid dynamic scene from a single mo...
research
07/05/2017

Robust Multi-Image HDR Reconstruction for the Modulo Camera

Photographing scenes with high dynamic range (HDR) poses great challenge...

Please sign up or login with your details

Forgot password? Click here to reset