DeepAI AI Chat
Log In Sign Up

Detachedly Learn a Classifier for Class-Incremental Learning

by   Ziheng Li, et al.
Peking University

In continual learning, model needs to continually learn a feature extractor and classifier on a sequence of tasks. This paper focuses on how to learn a classifier based on a pretrained feature extractor under continual learning setting. We present an probabilistic analysis that the failure of vanilla experience replay (ER) comes from unnecessary re-learning of previous tasks and incompetence to distinguish current task from the previous ones, which is the cause of knowledge degradation and prediction bias. To overcome these weaknesses, we propose a novel replay strategy task-aware experience replay. It rebalances the replay loss and detaches classifier weight for the old tasks from the update process, by which the previous knowledge is kept intact and the overfitting on episodic memory is alleviated. Experimental results show our method outperforms current state-of-the-art methods.


page 4

page 6

page 7


Continual Learning with Strong Experience Replay

Continual Learning (CL) aims at incrementally learning new tasks without...

Dealing with Cross-Task Class Discrimination in Online Continual Learning

Existing continual learning (CL) research regards catastrophic forgettin...

Generative Feature Replay For Class-Incremental Learning

Humans are capable of learning new tasks without forgetting previous one...

Towards Robust Inductive Graph Incremental Learning via Experience Replay

Inductive node-wise graph incremental learning is a challenging task due...

KRNet: Towards Efficient Knowledge Replay

The knowledge replay technique has been widely used in many tasks such a...

AttriCLIP: A Non-Incremental Learner for Incremental Knowledge Learning

Continual learning aims to enable a model to incrementally learn knowled...

SHARP: Sparsity and Hidden Activation RePlay for Neuro-Inspired Continual Learning

Deep neural networks (DNNs) struggle to learn in dynamic environments si...