HoW-3D: Holistic 3D Wireframe Perception from a Single Image

08/15/2022
by   Wenchao Ma, et al.
6

This paper studies the problem of holistic 3D wireframe perception (HoW-3D), a new task of perceiving both the visible 3D wireframes and the invisible ones from single-view 2D images. As the non-front surfaces of an object cannot be directly observed in a single view, estimating the non-line-of-sight (NLOS) geometries in HoW-3D is a fundamentally challenging problem and remains open in computer vision. We study the problem of HoW-3D by proposing an ABC-HoW benchmark, which is created on top of CAD models sourced from the ABC-dataset with 12k single-view images and the corresponding holistic 3D wireframe models. With our large-scale ABC-HoW benchmark available, we present a novel Deep Spatial Gestalt (DSG) model to learn the visible junctions and line segments as the basis and then infer the NLOS 3D structures from the visible cues by following the Gestalt principles of human vision systems. In our experiments, we demonstrate that our DSG model performs very well in inferring the holistic 3D wireframes from single-view images. Compared with the strong baseline methods, our DSG model outperforms the previous wireframe detectors in detecting the invisible line geometry in single-view images and is even very competitive with prior arts that take high-fidelity PointCloud as inputs on reconstructing 3D wireframes.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/17/2022

S^3-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint

In this paper, we address the "dual problem" of multi-view scene reconst...
research
09/06/2021

CTRL-C: Camera calibration TRansformer with Line-Classification

Single image camera calibration is the task of estimating the camera par...
research
07/26/2020

OASIS: A Large-Scale Dataset for Single Image 3D in the Wild

Single-view 3D is the task of recovering 3D properties such as depth and...
research
03/16/2016

DeepContext: Context-Encoding Neural Pathways for 3D Holistic Scene Understanding

While deep neural networks have led to human-level performance on comput...
research
10/08/2018

Novel Single View Constraints for Manhattan 3D Line Reconstruction

This paper proposes a novel and exact method to reconstruct line-based 3...
research
06/16/2022

Virtual Correspondence: Humans as a Cue for Extreme-View Geometry

Recovering the spatial layout of the cameras and the geometry of the sce...
research
03/05/2020

From Perspective X-ray Imaging to Parallax-Robust Orthographic Stitching

Stitching images acquired under perspective projective geometry is a rel...

Please sign up or login with your details

Forgot password? Click here to reset