All-pairs Consistency Learning for Weakly Supervised Semantic Segmentation

by   Weixuan Sun, et al.

In this work, we propose a new transformer-based regularization to better localize objects for Weakly supervised semantic segmentation (WSSS). In image-level WSSS, Class Activation Map (CAM) is adopted to generate object localization as pseudo segmentation labels. To address the partial activation issue of the CAMs, consistency regularization is employed to maintain activation intensity invariance across various image augmentations. However, such methods ignore pair-wise relations among regions within each CAM, which capture context and should also be invariant across image views. To this end, we propose a new all-pairs consistency regularization (ACR). Given a pair of augmented views, our approach regularizes the activation intensities between a pair of augmented views, while also ensuring that the affinity across regions within each view remains consistent. We adopt vision transformers as the self-attention mechanism naturally embeds pair-wise affinity. This enables us to simply regularize the distance between the attention matrices of augmented image pairs. Additionally, we introduce a novel class-wise localization method that leverages the gradients of the class token. Our method can be seamlessly integrated into existing WSSS methods using transformers without modifying the architectures. We evaluate our method on PASCAL VOC and MS COCO datasets. Our method produces noticeably better class localization maps (67.3 VOC train), resulting in superior WSSS performances.


page 6

page 7

page 8

page 15

page 16

page 17

page 18

page 19


Multi-class Token Transformer for Weakly Supervised Semantic Segmentation

This paper proposes a new transformer-based framework to learn class-spe...

Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformers

Weakly-supervised semantic segmentation (WSSS) with image-level labels i...

MCTformer+: Multi-Class Token Transformer for Weakly Supervised Semantic Segmentation

This paper proposes a novel transformer-based framework that aims to enh...

Token Contrast for Weakly-Supervised Semantic Segmentation

Weakly-Supervised Semantic Segmentation (WSSS) using image-level labels ...

Mitigating Undisciplined Over-Smoothing in Transformer for Weakly Supervised Semantic Segmentation

A surge of interest has emerged in weakly supervised semantic segmentati...

Attention-based Class Activation Diffusion for Weakly-Supervised Semantic Segmentation

Extracting class activation maps (CAM) is a key step for weakly-supervis...

Semantic-Constraint Matching Transformer for Weakly Supervised Object Localization

Weakly supervised object localization (WSOL) strives to learn to localiz...

Please sign up or login with your details

Forgot password? Click here to reset