CPCM: Contextual Point Cloud Modeling for Weakly-supervised Point Cloud Semantic Segmentation

by   Lizhao Liu, et al.

We study the task of weakly-supervised point cloud semantic segmentation with sparse annotations (e.g., less than 0.1 the expensive cost of dense annotations. Unfortunately, with extremely sparse annotated points, it is very difficult to extract both contextual and object information for scene understanding such as semantic segmentation. Motivated by masked modeling (e.g., MAE) in image and video representation learning, we seek to endow the power of masked modeling to learn contextual information from sparsely-annotated points. However, directly applying MAE to 3D point clouds with sparse annotations may fail to work. First, it is nontrivial to effectively mask out the informative visual context from 3D point clouds. Second, how to fully exploit the sparse annotations for context modeling remains an open question. In this paper, we propose a simple yet effective Contextual Point Cloud Modeling (CPCM) method that consists of two parts: a region-wise masking (RegionMask) strategy and a contextual masked training (CMT) method. Specifically, RegionMask masks the point cloud continuously in geometric space to construct a meaningful masked prediction task for subsequent context learning. CMT disentangles the learning of supervised segmentation and unsupervised masked context prediction for effectively learning the very limited labeled points and mass unlabeled points, respectively. Extensive experiments on the widely-tested ScanNet V2 and S3DIS benchmarks demonstrate the superiority of CPCM over the state-of-the-art.


page 1

page 3

page 5

page 8

page 13

page 14

page 15

page 19


Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds

Semantic segmentation on 3D point clouds is an important task for 3D sce...

PointMatch: A Consistency Training Framework for Weakly SupervisedSemantic Segmentation of 3D Point Clouds

Semantic segmentation of point cloud usually relies on dense annotation ...

Attentive Context Normalization for Robust Permutation-Equivariant Learning

Many problems in computer vision require dealing with sparse, unstructur...

Dual Adaptive Transformations for Weakly Supervised Point Cloud Segmentation

Weakly supervised point cloud segmentation, i.e. semantically segmenting...

Weakly Supervised Scene Parsing with Point-based Distance Metric Learning

Semantic scene parsing is suffering from the fact that pixel-level annot...

Hierarchical Point-based Active Learning for Semi-supervised Point Cloud Semantic Segmentation

Impressive performance on point cloud semantic segmentation has been ach...

GaIA: Graphical Information Gain based Attention Network for Weakly Supervised Point Cloud Semantic Segmentation

While point cloud semantic segmentation is a significant task in 3D scen...

Please sign up or login with your details

Forgot password? Click here to reset