Deep Polarization Imaging for 3D shape and SVBRDF Acquisition

05/06/2021
by   Valentin Deschaintre, et al.
5

We present a novel method for efficient acquisition of shape and spatially varying reflectance of 3D objects using polarization cues. Unlike previous works that have exploited polarization to estimate material or object appearance under certain constraints (known shape or multiview acquisition), we lift such restrictions by coupling polarization imaging with deep learning to achieve high quality estimate of 3D object shape (surface normals and depth) and SVBRDF using single-view polarization imaging under frontal flash illumination. In addition to acquired polarization images, we provide our deep network with strong novel cues related to shape and reflectance, in the form of a normalized Stokes map and an estimate of diffuse color. We additionally describe modifications to network architecture and training loss which provide further qualitative improvements. We demonstrate our approach to achieve superior results compared to recent works employing deep learning in conjunction with flash illumination.

READ FULL TEXT
POST COMMENT

Comments

marecki

Hello! Would you like to ask if you are going in the direction of AI drawing from "flat" photography to create a photo in 3D? Personally, I am looking forward to this day, when it will be possible to print photos in 3D, eg .MPO format, without special glasses. I am already collecting photographic material greetings Marek Sz. (translation: google)

Authors

page 3

page 4

page 5

page 7

page 8

09/06/2018

3D Surface Reconstruction by Pointillism

The objective of this work is to infer the 3D shape of an object from a ...
07/17/2020

3D Human Shape Reconstruction from a Polarization Image

This paper tackles the problem of estimating 3D body shape of clothed hu...
01/07/2022

NeROIC: Neural Rendering of Objects from Online Image Collections

We present a novel method to acquire object representations from online ...
04/16/2018

Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image

We propose a material acquisition approach to recover the spatially-vary...
06/27/2019

Flexible SVBRDF Capture with a Multi-Image Deep Network

Empowered by deep learning, recent methods for material capture can esti...
04/03/2018

Deep Appearance Maps

We propose a deep representation of appearance, i. e. the relation of co...
05/31/2018

Imaging with SPADs and DMDs: Seeing through Diffraction-Photons

This paper addresses the problem of imaging in the presence of diffracti...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.