SATS: Self-Attention Transfer for Continual Semantic Segmentation

03/15/2022
by   Yiqiao Qiu, et al.
0

Continually learning to segment more and more types of image regions is a desired capability for many intelligent systems. However, such continual semantic segmentation suffers from the same catastrophic forgetting issue as in continual classification learning. While multiple knowledge distillation strategies originally for continual classification have been well adapted to continual semantic segmentation, they only consider transferring old knowledge based on the outputs from one or more layers of deep fully convolutional networks. Different from existing solutions, this study proposes to transfer a new type of information relevant to knowledge, i.e. the relationships between elements (Eg. pixels or small local regions) within each image which can capture both within-class and between-class knowledge. The relationship information can be effectively obtained from the self-attention maps in a Transformer-style segmentation model. Considering that pixels belonging to the same class in each image often share similar visual properties, a class-specific region pooling is applied to provide more efficient relationship information for knowledge transfer. Extensive evaluations on multiple public benchmarks support that the proposed self-attention transfer method can further effectively alleviate the catastrophic forgetting issue, and its flexible combination with one or more widely adopted strategies significantly outperforms state-of-the-art solu

READ FULL TEXT

page 4

page 7

research
03/10/2022

Representation Compensation Networks for Continual Semantic Segmentation

In this work, we study the continual semantic segmentation problem, wher...
research
11/23/2020

PLOP: Learning without Forgetting for Continual Semantic Segmentation

Deep learning approaches are nowadays ubiquitously used to tackle comput...
research
11/25/2022

CoMFormer: Continual Learning in Semantic and Panoptic Segmentation

Continual learning for segmentation has recently seen increasing interes...
research
04/17/2022

Continual Hippocampus Segmentation with Transformers

In clinical settings, where acquisition conditions and patient populatio...
research
04/08/2023

Continual Learning for LiDAR Semantic Segmentation: Class-Incremental and Coarse-to-Fine strategies on Sparse Data

During the last few years, continual learning (CL) strategies for image ...
research
03/16/2022

RBC: Rectifying the Biased Context in Continual Semantic Segmentation

Recent years have witnessed a great development of Convolutional Neural ...
research
09/03/2021

Complementary Calibration: Boosting General Continual Learning with Collaborative Distillation and Self-Supervision

General Continual Learning (GCL) aims at learning from non independent a...

Please sign up or login with your details

Forgot password? Click here to reset