Neural Part Priors: Learning to Optimize Part-Based Object Completion in RGB-D Scans

03/17/2022
by   Alexey Bokhovkin, et al.
0

3D object recognition has seen significant advances in recent years, showing impressive performance on real-world 3D scan benchmarks, but lacking in object part reasoning, which is fundamental to higher-level scene understanding such as inter-object similarities or object functionality. Thus, we propose to leverage large-scale synthetic datasets of 3D shapes annotated with part information to learn Neural Part Priors (NPPs), optimizable spaces characterizing geometric part priors. Crucially, we can optimize over the learned part priors in order to fit to real-world scanned 3D scenes at test time, enabling robust part decomposition of the real objects in these scenes that also estimates the complete geometry of the object while fitting accurately to the observed real geometry. Moreover, this enables global optimization over geometrically similar detected objects in a scene, which often share strong geometric commonalities, enabling scene-consistent part decompositions. Experiments on the ScanNet dataset demonstrate that NPPs significantly outperforms state of the art in part decomposition and object completion in real-world scenes.

READ FULL TEXT

page 1

page 5

page 7

page 11

page 14

page 20

page 21

page 22

research
12/03/2020

Towards Part-Based Understanding of RGB-D Scans

Recent advances in 3D semantic scene understanding have shown impressive...
research
04/26/2019

3D-SIC: 3D Semantic Instance Completion for RGB-D Scans

This paper focuses on the task of semantic instance completion: from an ...
research
12/12/2022

ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenes

The two popular datasets ScanRefer [16] and ReferIt3D [3] connect natura...
research
10/19/2015

PERCH: Perception via Search for Multi-Object Recognition and Localization

In many robotic domains such as flexible automated manufacturing or pers...
research
08/24/2020

Interactive Annotation of 3D Object Geometry using 2D Scribbles

Inferring detailed 3D geometry of the scene is crucial for robotics appl...
research
09/21/2017

SceneCut: Joint Geometric and Object Segmentation for Indoor Scenes

This paper presents SceneCut, a novel approach to jointly discover previ...
research
10/05/2021

Mix3D: Out-of-Context Data Augmentation for 3D Scenes

We present Mix3D, a data augmentation technique for segmenting large-sca...

Please sign up or login with your details

Forgot password? Click here to reset