Log In Sign Up

Pre-Training by Completing Point Clouds

by   Hanchen Wang, et al.

There has recently been a flurry of exciting advances in deep learning models on point clouds. However, these advances have been hampered by the difficulty of creating labelled point cloud datasets: sparse point clouds often have unclear label identities for certain points, while dense point clouds are time-consuming to annotate. Inspired by mask-based pre-training in the natural language processing community, we propose a novel pre-training mechanism for point clouds. It works by masking occluded points that result from observing the point cloud at different camera views. It then optimizes a completion model that learns how to reconstruct the occluded points, given the partial point cloud. In this way, our method learns a pre-trained representation that can identify the visual constraints inherently embedded in real-world point clouds. We call our method Occlusion Completion (OcCo). We demonstrate that OcCo learns representations that improve generalization on downstream tasks over prior pre-training methods, that transfer to different datasets, that reduce training time, and improve labelled sample efficiency. previous pre-training methods. Our code and dataset are available at


POS-BERT: Point Cloud One-Stage BERT Pre-Training

Recently, the pre-training paradigm combining Transformer and masked lan...

SimIPU: Simple 2D Image and 3D Point Cloud Unsupervised Pre-Training for Spatial-Aware Visual Representations

Pre-training has become a standard paradigm in many computer vision task...

LassoNet: Deep Lasso-Selection of 3D Point Clouds

Selection is a fundamental task in exploratory analysis and visualizatio...

P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting

Nowadays, pre-training big models on large-scale datasets has become a c...

Learning to Generate Dense Point Clouds with Textures on Multiple Categories

3D reconstruction from images is a core problem in computer vision. With...

Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling

We identify and overcome two key obstacles in extending the success of B...

Completing point cloud from few points by Wasserstein GAN and Transformers

In many vision and robotics applications, it is common that the captured...