Image Understands Point Cloud: Weakly Supervised 3D Semantic Segmentation via Association Learning

09/16/2022
by   Tianfang Sun, et al.
0

Weakly supervised point cloud semantic segmentation methods that require 1% or fewer labels, hoping to realize almost the same performance as fully supervised approaches, which recently, have attracted extensive research attention. A typical solution in this framework is to use self-training or pseudo labeling to mine the supervision from the point cloud itself, but ignore the critical information from images. In fact, cameras widely exist in LiDAR scenarios and this complementary information seems to be greatly important for 3D applications. In this paper, we propose a novel cross-modality weakly supervised method for 3D segmentation, incorporating complementary information from unlabeled images. Basically, we design a dual-branch network equipped with an active labeling strategy, to maximize the power of tiny parts of labels and directly realize 2D-to-3D knowledge transfer. Afterwards, we establish a cross-modal self-training framework in an Expectation-Maximum (EM) perspective, which iterates between pseudo labels estimation and parameters updating. In the M-Step, we propose a cross-modal association learning to mine complementary supervision from images by reinforcing the cycle-consistency between 3D points and 2D superpixels. In the E-step, a pseudo label self-rectification mechanism is derived to filter noise labels thus providing more accurate labels for the networks to get fully trained. The extensive experimental results demonstrate that our method even outperforms the state-of-the-art fully supervised competitors with less than 1% actively selected annotations.

READ FULL TEXT

page 6

page 12

page 13

page 15

research
12/09/2022

Weakly Supervised Semantic Segmentation for Large-Scale Point Cloud

Existing methods for large-scale point cloud semantic segmentation requi...
research
04/06/2021

One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation

Point cloud semantic segmentation often requires largescale annotated tr...
research
05/07/2022

Automatic segmentation of meniscus based on MAE self-supervision and point-line weak supervision paradigm

Medical image segmentation based on deep learning is often faced with th...
research
03/09/2023

Reliability-Adaptive Consistency Regularization for Weakly-Supervised Point Cloud Segmentation

Weakly-supervised point cloud segmentation with extremely limited labels...
research
09/19/2023

Cross-modal and Cross-domain Knowledge Transfer for Label-free 3D Segmentation

Current state-of-the-art point cloud-based perception methods usually re...
research
11/23/2022

One Class One Click: Quasi Scene-level Weakly Supervised Point Cloud Semantic Segmentation with Active Learning

Reliance on vast annotations to achieve leading performance severely res...
research
11/02/2022

Hypergraph Convolutional Network based Weakly Supervised Point Cloud Semantic Segmentation with Scene-Level Annotations

Point cloud segmentation with scene-level annotations is a promising but...

Please sign up or login with your details

Forgot password? Click here to reset