Through the Looking Glass: Neural 3D Reconstruction of Transparent Shapes

04/22/2020
by   Zhengqin Li, et al.
16

Recovering the 3D shape of transparent objects using a small number of unconstrained natural images is an ill-posed problem. Complex light paths induced by refraction and reflection have prevented both traditional and deep multiview stereo from solving this challenge. We propose a physically-based network to recover 3D shape of transparent objects using a few images acquired with a mobile phone camera, under a known but arbitrary environment map. Our novel contributions include a normal representation that enables the network to model complex light transport through local computation, a rendering layer that models refractions and reflections, a cost volume specifically designed for normal refinement of transparent shapes and a feature mapping based on predicted normals for 3D point cloud reconstruction. We render a synthetic dataset to encourage the model to learn refractive light transport across different views. Our experiments show successful recovery of high-quality 3D geometry for complex transparent shapes using as few as 5-12 natural images. Code and data are publicly released.

READ FULL TEXT

page 1

page 2

page 4

page 7

page 8

page 10

page 11

page 12

research
08/25/2022

Polarimetric Inverse Rendering for Transparent Shapes Reconstruction

In this work, we propose a novel method for the detailed reconstruction ...
research
05/09/2018

Full 3D Reconstruction of Transparent Objects

Numerous techniques have been proposed for reconstructing 3D models for ...
research
03/23/2022

A Hybrid Mesh-neural Representation for 3D Transparent Object Reconstruction

We propose a novel method to reconstruct the 3D shapes of transparent ob...
research
11/12/2018

Deep-learning the Latent Space of Light Transport

We suggest a method to directly deep-learn light transport, i. e., the m...
research
04/12/2019

Evolved Art with Transparent, Overlapping, and Geometric Shapes

In this work, an evolutionary art project is presented where images are ...
research
02/02/2022

Eikonal Fields for Refractive Novel-View Synthesis

We tackle the problem of generating novel-view images from collections o...
research
03/24/2023

Seeing Through the Glass: Neural 3D Reconstruction of Object Inside a Transparent Container

In this paper, we define a new problem of recovering the 3D geometry of ...

Please sign up or login with your details

Forgot password? Click here to reset