Neural Photometric Stereo Reconstruction for General Reflectance Surfaces

02/28/2018
by   Tatsunori Taniai, et al.
0

We present a novel convolutional neural network architecture for photometric stereo (Woodham, 1980), a problem of recovering 3D object surface normals from multiple images observed under varying illuminations. Despite its long history in computer vision, the problem still shows fundamental challenges for surfaces with unknown general reflectance properties (BRDFs). Leveraging deep neural networks to learn complicated reflectance models is promising, but studies in this direction are very limited due to difficulties in acquiring accurate ground truth for training and also in designing networks invariant to permutation of input images. In order to address these challenges, we propose a reconstruction based unsupervised learning framework where surface normals and BRDFs are predicted by the network and fed into the rendering equation to synthesize observed images. The network is trained during testing by minimizing reconstruction loss between observed and synthesized images. Thus, our learning process does not require ground truth normals or even pre-training on external images. Our method is shown to achieve the state-of-the-art performance on a challenging real-world scene benchmark.

READ FULL TEXT

page 7

page 14

page 17

page 18

page 20

page 22

page 24

page 25

research
12/12/2020

Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfaces

This paper presents an uncalibrated deep neural network framework for th...
research
03/15/2021

Machine Learning for Nondestructive Wear Assessment in Large Internal Combustion Engines

Digitalization offers a large number of promising tools for large intern...
research
06/06/2022

Spatial Acoustic Projection for 3D Imaging Sonar Reconstruction

In this work we present a novel method for reconstructing 3D surfaces us...
research
07/05/2022

DeepPS2: Revisiting Photometric Stereo Using Two Differently Illuminated Images

Photometric stereo, a problem of recovering 3D surface normals using ima...
research
07/15/2021

Incorporating Lambertian Priors into Surface Normals Measurement

The goal of photometric stereo is to measure the precise surface normal ...
research
03/24/2022

Neural Reflectance for Shape Recovery with Shadow Handling

This paper aims at recovering the shape of a scene with unknown, non-Lam...
research
03/28/2022

DeepShadow: Neural Shape from Shadow

This paper presents DeepShadow, a one-shot method for recovering the dep...

Please sign up or login with your details

Forgot password? Click here to reset