DeepAI AI Chat
Log In Sign Up

Detachedly Learn a Classifier for Class-Incremental Learning

02/23/2023
by   Ziheng Li, et al.
Peking University
0

In continual learning, model needs to continually learn a feature extractor and classifier on a sequence of tasks. This paper focuses on how to learn a classifier based on a pretrained feature extractor under continual learning setting. We present an probabilistic analysis that the failure of vanilla experience replay (ER) comes from unnecessary re-learning of previous tasks and incompetence to distinguish current task from the previous ones, which is the cause of knowledge degradation and prediction bias. To overcome these weaknesses, we propose a novel replay strategy task-aware experience replay. It rebalances the replay loss and detaches classifier weight for the old tasks from the update process, by which the previous knowledge is kept intact and the overfitting on episodic memory is alleviated. Experimental results show our method outperforms current state-of-the-art methods.

READ FULL TEXT

page 4

page 6

page 7

05/23/2023

Continual Learning with Strong Experience Replay

Continual Learning (CL) aims at incrementally learning new tasks without...
05/24/2023

Dealing with Cross-Task Class Discrimination in Online Continual Learning

Existing continual learning (CL) research regards catastrophic forgettin...
04/20/2020

Generative Feature Replay For Class-Incremental Learning

Humans are capable of learning new tasks without forgetting previous one...
02/07/2023

Towards Robust Inductive Graph Incremental Learning via Experience Replay

Inductive node-wise graph incremental learning is a challenging task due...
05/23/2022

KRNet: Towards Efficient Knowledge Replay

The knowledge replay technique has been widely used in many tasks such a...
05/19/2023

AttriCLIP: A Non-Incremental Learner for Incremental Knowledge Learning

Continual learning aims to enable a model to incrementally learn knowled...
05/29/2023

SHARP: Sparsity and Hidden Activation RePlay for Neuro-Inspired Continual Learning

Deep neural networks (DNNs) struggle to learn in dynamic environments si...