SemFormer: Semantic Guided Activation Transformer for Weakly Supervised Semantic Segmentation

by   Junliang Chen, et al.
Shenzhen University

Recent mainstream weakly supervised semantic segmentation (WSSS) approaches are mainly based on Class Activation Map (CAM) generated by a CNN (Convolutional Neural Network) based image classifier. In this paper, we propose a novel transformer-based framework, named Semantic Guided Activation Transformer (SemFormer), for WSSS. We design a transformer-based Class-Aware AutoEncoder (CAAE) to extract the class embeddings for the input image and learn class semantics for all classes of the dataset. The class embeddings and learned class semantics are then used to guide the generation of activation maps with four losses, i.e., class-foreground, class-background, activation suppression, and activation complementation loss. Experimental results show that our SemFormer achieves 74.3% mIoU and surpasses many recent mainstream WSSS approaches by a large margin on PASCAL VOC 2012 dataset. Code will be available at <>.


page 3

page 6

page 9

page 13

page 14


SLAM: Semantic Learning based Activation Map for Weakly Supervised Semantic Segmentation

Recent mainstream weakly-supervised semantic segmentation (WSSS) approac...

WeakTr: Exploring Plain Vision Transformer for Weakly-supervised Semantic Segmentation

This paper explores the properties of the plain Vision Transformer (ViT)...

Weakly-Supervised Image Semantic Segmentation Using Graph Convolutional Networks

This work addresses weakly-supervised image semantic segmentation based ...

Boosting Semantic Segmentation from the Perspective of Explicit Class Embeddings

Semantic segmentation is a computer vision task that associates a label ...

Class Activation Map generation by Multiple Level Class Grouping and Orthogonal Constraint

Class activation map (CAM) highlights regions of classes based on classi...

L2G: A Simple Local-to-Global Knowledge Transfer Framework for Weakly Supervised Semantic Segmentation

Mining precise class-aware attention maps, a.k.a, class activation maps,...

Cross Language Image Matching for Weakly Supervised Semantic Segmentation

It has been widely known that CAM (Class Activation Map) usually only ac...

Please sign up or login with your details

Forgot password? Click here to reset