DäRF: Boosting Radiance Fields from Sparse Inputs with Monocular Depth Adaptation

by   Jiuhn Song, et al.

Neural radiance fields (NeRF) shows powerful performance in novel view synthesis and 3D geometry reconstruction, but it suffers from critical performance degradation when the number of known viewpoints is drastically reduced. Existing works attempt to overcome this problem by employing external priors, but their success is limited to certain types of scenes or datasets. Employing monocular depth estimation (MDE) networks, pretrained on large-scale RGB-D datasets, with powerful generalization capability would be a key to solving this problem: however, using MDE in conjunction with NeRF comes with a new set of challenges due to various ambiguity problems exhibited by monocular depths. In this light, we propose a novel framework, dubbed DäRF, that achieves robust NeRF reconstruction with a handful of real-world images by combining the strengths of NeRF and monocular depth estimation through online complementary training. Our framework imposes the MDE network's powerful geometry prior to NeRF representation at both seen and unseen viewpoints to enhance its robustness and coherence. In addition, we overcome the ambiguity problems of monocular depths through patch-wise scale-shift fitting and geometry distillation, which adapts the MDE network to produce depths aligned accurately with NeRF geometry. Experiments show our framework achieves state-of-the-art results both quantitatively and qualitatively, demonstrating consistent and reliable performance in both indoor and outdoor real-world datasets. Project page is available at https://ku-cvlab.github.io/DaRF/.


page 18

page 19

page 20

page 21

page 22

page 23

page 24

page 25


SCADE: NeRFs from Space Carving with Ambiguity-Aware Depth Estimates

Neural radiance fields (NeRFs) have enabled high fidelity 3D reconstruct...

NVS-MonoDepth: Improving Monocular Depth Prediction with Novel View Synthesis

Building upon the recent progress in novel view synthesis, we propose it...

NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo

In this work, we present a new multi-view depth estimation method that u...

Crafting Monocular Cues and Velocity Guidance for Self-Supervised Multi-Frame Depth Learning

Self-supervised monocular methods can efficiently learn depth informatio...

NDDepth: Normal-Distance Assisted Monocular Depth Estimation

Monocular depth estimation has drawn widespread attention from the visio...

Robust Geometry-Preserving Depth Estimation Using Differentiable Rendering

In this study, we address the challenge of 3D scene structure recovery f...

360^∘ Reconstruction From a Single Image Using Space Carved Outpainting

We introduce POP3D, a novel framework that creates a full 360^∘-view 3D ...

Please sign up or login with your details

Forgot password? Click here to reset