DeepAI
Log In Sign Up

Pre-Training by Completing Point Clouds

10/02/2020
by   Hanchen Wang, et al.
1

There has recently been a flurry of exciting advances in deep learning models on point clouds. However, these advances have been hampered by the difficulty of creating labelled point cloud datasets: sparse point clouds often have unclear label identities for certain points, while dense point clouds are time-consuming to annotate. Inspired by mask-based pre-training in the natural language processing community, we propose a novel pre-training mechanism for point clouds. It works by masking occluded points that result from observing the point cloud at different camera views. It then optimizes a completion model that learns how to reconstruct the occluded points, given the partial point cloud. In this way, our method learns a pre-trained representation that can identify the visual constraints inherently embedded in real-world point clouds. We call our method Occlusion Completion (OcCo). We demonstrate that OcCo learns representations that improve generalization on downstream tasks over prior pre-training methods, that transfer to different datasets, that reduce training time, and improve labelled sample efficiency. previous pre-training methods. Our code and dataset are available at https://github.com/hansen7/OcCo

READ FULL TEXT
04/03/2022

POS-BERT: Point Cloud One-Stage BERT Pre-Training

Recently, the pre-training paradigm combining Transformer and masked lan...
12/09/2021

SimIPU: Simple 2D Image and 3D Point Cloud Unsupervised Pre-Training for Spatial-Aware Visual Representations

Pre-training has become a standard paradigm in many computer vision task...
07/31/2019

LassoNet: Deep Lasso-Selection of 3D Point Clouds

Selection is a fundamental task in exploratory analysis and visualizatio...
08/04/2022

P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting

Nowadays, pre-training big models on large-scale datasets has become a c...
12/22/2019

Learning to Generate Dense Point Clouds with Textures on Multiple Categories

3D reconstruction from images is a core problem in computer vision. With...
01/09/2023

Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling

We identify and overcome two key obstacles in extending the success of B...
11/23/2022

Completing point cloud from few points by Wasserstein GAN and Transformers

In many vision and robotics applications, it is common that the captured...