FineRecon: Depth-aware Feed-forward Network for Detailed 3D Reconstruction

04/04/2023
by   Noah Stier, et al.
0

Recent works on 3D reconstruction from posed images have demonstrated that direct inference of scene-level 3D geometry without iterative optimization is feasible using a deep neural network, showing remarkable promise and high efficiency. However, the reconstructed geometries, typically represented as a 3D truncated signed distance function (TSDF), are often coarse without fine geometric details. To address this problem, we propose three effective solutions for improving the fidelity of inference-based 3D reconstructions. We first present a resolution-agnostic TSDF supervision strategy to provide the network with a more accurate learning signal during training, avoiding the pitfalls of TSDF interpolation seen in previous work. We then introduce a depth guidance strategy using multi-view depth estimates to enhance the scene representation and recover more accurate surfaces. Finally, we develop a novel architecture for the final layers of the network, conditioning the output TSDF prediction on high-resolution image features in addition to coarse voxel features, enabling sharper reconstruction of fine details. Our method produces smooth and highly accurate reconstructions, showing significant improvements across multiple depth and 3D reconstruction metrics.

READ FULL TEXT

page 1

page 5

page 7

research
12/11/2020

Detailed 3D Human Body Reconstruction from Multi-view Images Combining Voxel Super-Resolution and Learned Implicit Representation

The task of reconstructing detailed 3D human body models from images is ...
research
02/14/2023

HR-NeuS: Recovering High-Frequency Surface Geometry via Neural Implicit Surfaces

Recent advances in neural implicit surfaces for multi-view 3D reconstruc...
research
06/05/2023

Neuralangelo: High-Fidelity Neural Surface Reconstruction

Neural surface reconstruction has been shown to be powerful for recoveri...
research
05/25/2022

Efficient Textured Mesh Recovery from Multiple Views with Differentiable Rendering

Despite of the promising results on shape and color recovery using self-...
research
04/12/2023

Dynamic Voxel Grid Optimization for High-Fidelity RGB-D Supervised Surface Reconstruction

Direct optimization of interpolated features on multi-resolution voxel g...
research
11/27/2019

Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness

We present Uncertainty-aware Cascaded Stereo Network (UCS-Net) for 3D re...
research
06/15/2022

Subsurface Depths Structure Maps Reconstruction with Generative Adversarial Networks

This paper described a method for reconstruction of detailed-resolution ...

Please sign up or login with your details

Forgot password? Click here to reset