3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions

03/27/2016
by   Andy Zeng, et al.
0

Matching local geometric features on real-world depth images is a challenging task due to the noisy, low-resolution, and incomplete nature of 3D scan data. These difficulties limit the performance of current state-of-art methods, which are typically based on histograms over geometric properties. In this paper, we present 3DMatch, a data-driven model that learns a local volumetric patch descriptor for establishing correspondences between partial 3D data. To amass training data for our model, we propose a self-supervised feature learning method that leverages the millions of correspondence labels found in existing RGB-D reconstructions. Experiments show that our descriptor is not only able to match local geometry in new scenes for reconstruction, but also generalize to different tasks and spatial scales (e.g. instance-level object model alignment for the Amazon Picking Challenge, and mesh surface correspondence). Results show that 3DMatch consistently outperforms other state-of-the-art approaches by a significant margin. Code, data, benchmarks, and pre-trained models are available online at http://3dmatch.cs.princeton.edu

READ FULL TEXT

page 1

page 2

page 5

page 6

page 7

page 8

page 9

page 11

research
01/24/2017

Improved Descriptors for Patch Matching and Reconstruction

We propose a convolutional neural network (ConvNet) based approach for l...
research
11/29/2019

SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scans

We present a novel approach that converts partial and noisy RGB-D scans ...
research
11/27/2018

Scan2CAD: Learning CAD Model Alignment in RGB-D Scans

We present Scan2CAD, a novel data-driven method that learns to align cle...
research
06/25/2020

SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scans

We present SPSG, a novel approach to generate high-quality, colored 3D m...
research
12/13/2014

Descriptor Ensemble: An Unsupervised Approach to Descriptor Fusion in the Homography Space

With the aim to improve the performance of feature matching, we present ...
research
03/22/2018

PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D Reconstruction

We introduce a novel RGB-D patch descriptor designed for detecting copla...
research
08/04/2018

Learning to Align Images using Weak Geometric Supervision

Image alignment tasks require accurate pixel correspondences, which are ...

Please sign up or login with your details

Forgot password? Click here to reset