Leveraging Hidden Positives for Unsupervised Semantic Segmentation

03/27/2023
by   Hyun Seok Seong, et al.
0

Dramatic demand for manpower to label pixel-level annotations triggered the advent of unsupervised semantic segmentation. Although the recent work employing the vision transformer (ViT) backbone shows exceptional performance, there is still a lack of consideration for task-specific training guidance and local semantic consistency. To tackle these issues, we leverage contrastive learning by excavating hidden positives to learn rich semantic relationships and ensure semantic consistency in local regions. Specifically, we first discover two types of global hidden positives, task-agnostic and task-specific ones for each anchor based on the feature similarities defined by a fixed pre-trained backbone and a segmentation head-in-training, respectively. A gradual increase in the contribution of the latter induces the model to capture task-specific semantic features. In addition, we introduce a gradient propagation strategy to learn semantic consistency between adjacent patches, under the inherent premise that nearby patches are highly likely to possess the same semantics. Specifically, we add the loss propagating to local hidden positives, semantically similar nearby patches, in proportion to the predefined similarity scores. With these training schemes, our proposed method achieves new state-of-the-art (SOTA) results in COCO-stuff, Cityscapes, and Potsdam-3 datasets. Our code is available at: https://github.com/hynnsk/HP.

READ FULL TEXT

page 1

page 3

page 5

page 7

page 8

research
04/07/2022

L2G: A Simple Local-to-Global Knowledge Transfer Framework for Weakly Supervised Semantic Segmentation

Mining precise class-aware attention maps, a.k.a, class activation maps,...
research
03/30/2021

PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering

We present a new framework for semantic segmentation without annotations...
research
03/25/2023

IFSeg: Image-free Semantic Segmentation via Vision-Language Model

Vision-language (VL) pre-training has recently gained much attention for...
research
06/28/2023

GraSS: Contrastive Learning with Gradient Guided Sampling Strategy for Remote Sensing Image Semantic Segmentation

Self-supervised contrastive learning (SSCL) has achieved significant mil...
research
05/15/2023

Not All Pixels Are Equal: Learning Pixel Hardness for Semantic Segmentation

Semantic segmentation has recently witnessed great progress. Despite the...
research
08/23/2020

Robust Vision Challenge 2020 – 1st Place Report for Panoptic Segmentation

In this technical report, we present key details of our winning panoptic...
research
06/08/2019

Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentation

In this paper, we propose a novel Pattern-Affinitive Propagation (PAP) f...

Please sign up or login with your details

Forgot password? Click here to reset