Deep Photometric Stereo for Non-Lambertian Surfaces

by   Guanying Chen, et al.

This paper addresses the problem of photometric stereo, in both calibrated and uncalibrated scenarios, for non-Lambertian surfaces based on deep learning. We first introduce a fully convolutional deep network for calibrated photometric stereo, which we call PS-FCN. Unlike traditional approaches that adopt simplified reflectance models to make the problem tractable, our method directly learns the mapping from reflectance observations to surface normal, and is able to handle surfaces with general and unknown isotropic reflectance. At test time, PS-FCN takes an arbitrary number of images and their associated light directions as input and predicts a surface normal map of the scene in a fast feed-forward pass. To deal with the uncalibrated scenario where light directions are unknown, we introduce a new convolutional network, named LCNet, to estimate light directions from input images. The estimated light directions and the input images are then fed to PS-FCN to determine the surface normals. Our method does not require a pre-defined set of light directions and can handle multiple images in an order-agnostic manner. Thorough evaluation of our approach on both synthetic and real datasets shows that it outperforms state-of-the-art methods in both calibrated and uncalibrated scenarios.


page 2

page 5

page 6

page 7

page 8

page 9

page 10

page 12


PS-FCN: A Flexible Learning Framework for Photometric Stereo

This paper addresses the problem of photometric stereo for non-Lambertia...

Self-calibrating Deep Photometric Stereo Networks

This paper proposes an uncalibrated photometric stereo method for non-La...

CNN-PS: CNN-based Photometric Stereo for General Non-Convex Surfaces

Most conventional photometric stereo algorithms inversely solve a BRDF-b...

An Optimal Experimental Design Approach for Light Configurations in Photometric Stereo

This paper presents a technique for finding the surface normal of an obj...

Leveraging Spatial and Photometric Context for Calibrated Non-Lambertian Photometric Stereo

The problem of estimating a surface shape from its observed reflectance ...

Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfaces

This paper presents an uncalibrated deep neural network framework for th...

Neural apparent BRDF fields for multiview photometric stereo

We propose to tackle the multiview photometric stereo problem using an e...