PCR: Proxy-based Contrastive Replay for Online Class-Incremental Continual Learning

04/10/2023
by   Huiwei Lin, et al.
0

Online class-incremental continual learning is a specific task of continual learning. It aims to continuously learn new classes from data stream and the samples of data stream are seen only once, which suffers from the catastrophic forgetting issue, i.e., forgetting historical knowledge of old classes. Existing replay-based methods effectively alleviate this issue by saving and replaying part of old data in a proxy-based or contrastive-based replay manner. Although these two replay manners are effective, the former would incline to new classes due to class imbalance issues, and the latter is unstable and hard to converge because of the limited number of samples. In this paper, we conduct a comprehensive analysis of these two replay manners and find that they can be complementary. Inspired by this finding, we propose a novel replay-based method called proxy-based contrastive replay (PCR). The key operation is to replace the contrastive samples of anchors with corresponding proxies in the contrastive-based way. It alleviates the phenomenon of catastrophic forgetting by effectively addressing the imbalance issue, as well as keeps a faster convergence of the model. We conduct extensive experiments on three real-world benchmark datasets, and empirical results consistently demonstrate the superiority of PCR over various state-of-the-art methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/24/2023

Dealing with Cross-Task Class Discrimination in Online Continual Learning

Existing continual learning (CL) research regards catastrophic forgettin...
research
05/12/2023

RepCL: Exploring Effective Representation for Continual Text Classification

Continual learning (CL) aims to constantly learn new knowledge over time...
research
09/08/2023

UER: A Heuristic Bias Addressing Approach for Online Continual Learning

Online continual learning aims to continuously train neural networks fro...
research
09/18/2023

DFIL: Deepfake Incremental Learning by Exploiting Domain-invariant Forgery Clues

The malicious use and widespread dissemination of deepfake pose a signif...
research
09/03/2023

Efficient Curriculum based Continual Learning with Informative Subset Selection for Remote Sensing Scene Classification

We tackle the problem of class incremental learning (CIL) in the realm o...
research
08/18/2023

Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuning

Continual learning aims to learn a model from a continuous stream of dat...
research
09/18/2023

CaT: Balanced Continual Graph Learning with Graph Condensation

Continual graph learning (CGL) is purposed to continuously update a grap...

Please sign up or login with your details

Forgot password? Click here to reset