Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation

04/09/2020
by   Yude Wang, et al.
52

Image-level weakly supervised semantic segmentation is a challenging problem that has been deeply studied in recent years. Most of advanced solutions exploit class activation map (CAM). However, CAMs can hardly serve as the object mask due to the gap between full and weak supervisions. In this paper, we propose a self-supervised equivariant attention mechanism (SEAM) to discover additional supervision and narrow the gap. Our method is based on the observation that equivariance is an implicit constraint in fully supervised semantic segmentation, whose pixel-level labels take the same spatial transformation as the input images during data augmentation. However, this constraint is lost on the CAMs trained by image-level supervision. Therefore, we propose consistency regularization on predicted CAMs from various transformed images to provide self-supervision for network learning. Moreover, we propose a pixel correlation module (PCM), which exploits context appearance information and refines the prediction of current pixel by its similar neighbors, leading to further improvement on CAMs consistency. Extensive experiments on PASCAL VOC 2012 dataset demonstrate our method outperforms state-of-the-art methods using the same level of supervision. The code is released online.

READ FULL TEXT

page 1

page 3

page 6

page 8

research
09/09/2019

Self-supervised Scale Equivariant Network for Weakly Supervised Semantic Segmentation

Weakly supervised semantic segmentation has attracted much research inte...
research
10/12/2019

Saliency Guided Self-attention Network for Weakly-supervised Semantic Segmentation

Weakly supervised semantic segmentation (WSSS) using only image-level la...
research
12/10/2021

Exploring Pixel-level Self-supervision for Weakly Supervised Semantic Segmentation

Existing studies in weakly supervised semantic segmentation (WSSS) have ...
research
10/14/2021

Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

Though image-level weakly supervised semantic segmentation (WSSS) has ac...
research
04/06/2021

Weakly supervised segmentation with cross-modality equivariant constraints

Weakly supervised learning has emerged as an appealing alternative to al...
research
09/28/2021

Weakly Supervised Keypoint Discovery

In this paper, we propose a method for keypoint discovery from a 2D imag...
research
01/19/2022

Weakly Supervised Semantic Segmentation of Remote Sensing Images for Tree Species Classification Based on Explanation Methods

The collection of a high number of pixel-based labeled training samples ...

Please sign up or login with your details

Forgot password? Click here to reset