DeepAI
Log In Sign Up

Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation

06/29/2021
by   Arthur Douillard, et al.
25

Deep learning approaches are nowadays ubiquitously used to tackle computer vision tasks such as semantic segmentation, requiring large datasets and substantial computational power. Continual learning for semantic segmentation (CSS) is an emerging trend that consists in updating an old model by sequentially adding new classes. However, continual learning methods are usually prone to catastrophic forgetting. This issue is further aggravated in CSS where, at each step, old classes from previous iterations are collapsed into the background. In this paper, we propose Local POD, a multi-scale pooling distillation scheme that preserves long- and short-range spatial relationships at feature level. Furthermore, we design an entropy-based pseudo-labelling of the background w.r.t. classes predicted by the old model to deal with background shift and avoid catastrophic forgetting of the old classes. Finally, we introduce a novel rehearsal method that is particularly suited for segmentation. Our approach, called PLOP, significantly outperforms state-of-the-art methods in existing CSS scenarios, as well as in newly proposed challenging benchmarks.

READ FULL TEXT

page 4

page 6

page 12

page 13

11/23/2020

PLOP: Learning without Forgetting for Continual Semantic Segmentation

Deep learning approaches are nowadays ubiquitously used to tackle comput...
11/25/2022

CoMFormer: Continual Learning in Semantic and Panoptic Segmentation

Continual learning for segmentation has recently seen increasing interes...
11/13/2022

Mining Unseen Classes via Regional Objectness: A Simple Baseline for Incremental Segmentation

Incremental or continual learning has been extensively studied for image...
11/18/2022

Delving into Transformer for Incremental Semantic Segmentation

Incremental semantic segmentation(ISS) is an emerging task where old mod...
01/04/2022

Weakly-supervised continual learning for class-incremental segmentation

Transfer learning is a powerful way to adapt existing deep learning mode...
03/15/2022

SATS: Self-Attention Transfer for Continual Semantic Segmentation

Continually learning to segment more and more types of image regions is ...
08/08/2021

RECALL: Replay-based Continual Learning in Semantic Segmentation

Deep networks allow to obtain outstanding results in semantic segmentati...