Pano Popups: Indoor 3D Reconstruction with a Plane-Aware Network

07/01/2019
by   Marc Eder, et al.
3

In this work we present a method to train a plane-aware convolutional neural network for dense depth and surface normal estimation as well as plane boundaries from a single indoor image. Using our proposed loss function, our network outperforms existing methods for single-view, indoor, omnidirectional depth estimation and provides an initial benchmark for surface normal prediction from images. Our improvements are due to the use of a novel plane-aware loss that leverages principal curvature as an indicator of planar boundaries. We also show that including geodesic coordinate maps as network priors provides a significant boost in surface normal prediction accuracy. Finally, we demonstrate how we can combine our network's outputs to generate high quality 3D "pop-up" models of indoor scenes.

READ FULL TEXT

page 1

page 4

page 7

page 8

page 11

page 12

page 13

research
08/19/2021

StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimation

Self-supervised monocular depth estimation has achieved impressive perfo...
research
12/12/2017

Im2Pano3D: Extrapolating 360 Structure and Semantics Beyond the Field of View

We present Im2Pano3D, a convolutional neural network that generates a de...
research
12/13/2020

GeoNet++: Iterative Geometric Neural Network with Edge-Aware Refinement for Joint Depth and Surface Normal Estimation

In this paper, we propose a geometric neural network with edge-aware ref...
research
04/06/2021

Depth Completion with Twin Surface Extrapolation at Occlusion Boundaries

Depth completion starts from a sparse set of known depth values and esti...
research
06/27/2021

Indoor Panorama Planar 3D Reconstruction via Divide and Conquer

Indoor panorama typically consists of human-made structures parallel or ...
research
06/15/2022

Neural Network Normal Estimation and Bathymetry Reconstruction from Sidescan Sonar

Sidescan sonar intensity encodes information about the changes of surfac...
research
11/10/2022

Normal reconstruction from specularity in the endoscopic setting

We show that for a plane imaged by an endoscope the specular isophotes a...

Please sign up or login with your details

Forgot password? Click here to reset