Holistic 3D Scene Understanding from a Single Image with Implicit Representation

03/11/2021 ∙ by Cheng Zhang, et al. ∙ 47

We present a new pipeline for holistic 3D scene understanding from a single image, which could predict object shape, object pose, and scene layout. As it is a highly ill-posed problem, existing methods usually suffer from inaccurate estimation of both shapes and layout especially for the cluttered scene due to the heavy occlusion between objects. We propose to utilize the latest deep implicit representation to solve this challenge. We not only propose an image-based local structured implicit network to improve the object shape estimation, but also refine 3D object pose and scene layout via a novel implicit scene graph neural network that exploits the implicit local object features. A novel physical violation loss is also proposed to avoid incorrect context between objects. Extensive experiments demonstrate that our method outperforms the state-of-the-art methods in terms of object shape, scene layout estimation, and 3D object detection.

READ FULL TEXT
POST COMMENT

Comments

Authors

page 1

page 3

page 5

page 7

page 8

Code Repositories

Implicit3DUnderstanding

Official PyTorch code of Holistic 3D Scene Understanding from a Single Image with Implicit Representation (CVPR 2021). Also includes a PyTorch implementation of the decoder of LDIF (from 3D Shape Representation with Local Deep Implicit Functions).


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.