Improving neural implicit surfaces geometry with patch warping

12/17/2021
by   François Darmon, et al.
17

Neural implicit surfaces have become an important technique for multi-view 3D reconstruction but their accuracy remains limited. In this paper, we argue that this comes from the difficulty to learn and render high frequency textures with neural networks. We thus propose to add to the standard neural rendering optimization a direct photo-consistency term across the different views. Intuitively, we optimize the implicit geometry so that it warps views on each other in a consistent way. We demonstrate that two elements are key to the success of such an approach: (i) warping entire patches, using the predicted occupancy and normals of the 3D points along each ray, and measuring their similarity with a robust structural similarity (SSIM); (ii) handling visibility and occlusion in such a way that incorrect warps are not given too much importance while encouraging a reconstruction as complete as possible. We evaluate our approach, dubbed NeuralWarp, on the standard DTU and EPFL benchmarks and show it outperforms state of the art unsupervised implicit surfaces reconstructions by over 20

READ FULL TEXT

page 1

page 3

page 6

page 7

page 8

page 12

page 13

page 14

research
05/31/2022

Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction

Recently, neural implicit surfaces learning by volume rendering has beco...
research
03/30/2023

S-VolSDF: Sparse Multi-View Stereo Regularization of Neural Implicit Surfaces

Neural rendering of implicit surfaces performs well in 3D vision applica...
research
08/21/2023

Coordinate Quantized Neural Implicit Representations for Multi-view Reconstruction

In recent years, huge progress has been made on learning neural implicit...
research
11/26/2022

PatchShading: High-Quality Human Reconstruction by Patch Warping and Shading Refinement

Human reconstruction from multi-view images plays an important role in m...
research
06/22/2021

Volume Rendering of Neural Implicit Surfaces

Neural volume rendering became increasingly popular recently due to its ...
research
11/22/2022

Level-S^2fM: Structure from Motion on Neural Level Set of Implicit Surfaces

This paper presents a neural incremental Structure-from-Motion (SfM) app...
research
03/26/2017

Learned Multi-Patch Similarity

Estimating a depth map from multiple views of a scene is a fundamental t...

Please sign up or login with your details

Forgot password? Click here to reset