Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo

10/11/2021
by   Berk Kaya, et al.
0

We present a modern solution to the multi-view photometric stereo problem (MVPS). Our work suitably exploits the image formation model in a MVPS experimental setup to recover the dense 3D reconstruction of an object from images. We procure the surface orientation using a photometric stereo (PS) image formation model and blend it with a multi-view neural radiance field representation to recover the object's surface geometry. Contrary to the previous multi-staged framework to MVPS, where the position, iso-depth contours, or orientation measurements are estimated independently and then fused later, our method is simple to implement and realize. Our method performs neural rendering of multi-view images while utilizing surface normals estimated by a deep photometric stereo network. We render the MVPS images by considering the object's surface normals for each 3D sample point along the viewing direction rather than explicitly using the density gradient in the volume space via 3D occupancy information. We optimize the proposed neural radiance field representation for the MVPS setup efficiently using a fully connected deep network to recover the 3D geometry of an object. Extensive evaluation on the DiLiGenT-MV benchmark dataset shows that our method performs better than the approaches that perform only PS or only multi-view stereo (MVS) and provides comparable results against the state-of-the-art multi-stage fusion methods.

READ FULL TEXT

page 6

page 8

page 17

research
03/29/2023

Multi-View Azimuth Stereo via Tangent Space Consistency

We present a method for 3D reconstruction only using calibrated multi-vi...
research
10/14/2022

Multi-View Photometric Stereo Revisited

Multi-view photometric stereo (MVPS) is a preferred method for detailed ...
research
01/18/2020

Multi-View Photometric Stereo: A Robust Solution and Benchmark Dataset for Spatially Varying Isotropic Materials

We present a method to capture both 3D shape and spatially varying refle...
research
04/08/2022

Investigating Spherical Epipolar Rectification for Multi-View Stereo 3D Reconstruction

Multi-view stereo (MVS) reconstruction is essential for creating 3D mode...
research
02/26/2022

Uncertainty-Aware Deep Multi-View Photometric Stereo

This paper presents a simple and effective solution to the problem of mu...
research
07/25/2022

nLMVS-Net: Deep Non-Lambertian Multi-View Stereo

We introduce a novel multi-view stereo (MVS) method that can simultaneou...
research
10/09/2022

Estimating Neural Reflectance Field from Radiance Field using Tree Structures

We present a new method for estimating the Neural Reflectance Field (NRe...

Please sign up or login with your details

Forgot password? Click here to reset