A Simple Class Decision Balancing for Incremental Learning

03/31/2020
by   Hongjoon Ahn, et al.
0

Class incremental learning (CIL) problem, in which a learning agent continuously learns new classes from incrementally arriving training data batches, has gained much attention recently in AI and computer vision community due to both fundamental and practical perspectives of the problem. For mitigating the main difficulty of deep neural network(DNN)-based CIL, the catastrophic forgetting, recent work showed that a simple fine-tuning (FT) based schemes can outperform the earlier attempts of using knowledge distillation, particularly when a small-sized exemplar-memory for storing samples from the previously learned classes is allowed. The core limitation of the vanilla FT, however, is the severe classification score bias between the new and previously learned classes, and several state-of-the-art methods proposed to rectify the bias via additional post-processing of the scores. In this paper, we propose two simple modifications for the vanilla FT, separated softmax (SS) layer and ratio-preserving (RP) mini-batches for SGD updates. Our scheme, dubbed as SS-IL, is shown to give much more balanced class decisions, have much less biased scores, and outperform strong state-of-the-art baselines on several large-scale benchmark datasets, without any sophisticated post-processing of the scores. We also give several novel analyses our and baseline methods, confirming the effectiveness of our approach in CIL.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/04/2023

Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learning

Deep Neural Network (DNN) has achieved great success on datasets of clos...
research
01/16/2020

ScaIL: Classifier Weights Scaling for Class Incremental Learning

Incremental learning is useful if an AI agent needs to integrate data fr...
research
09/11/2023

MultIOD: Rehearsal-free Multihead Incremental Object Detector

Class-Incremental learning (CIL) is the ability of artificial agents to ...
research
08/31/2020

Initial Classifier Weights Replay for Memoryless Class Incremental Learning

Incremental Learning (IL) is useful when artificial systems need to deal...
research
05/30/2019

Large Scale Incremental Learning

Modern machine learning suffers from catastrophic forgetting when learni...
research
11/30/2020

Incremental Learning via Rate Reduction

Current deep learning architectures suffer from catastrophic forgetting,...
research
05/27/2022

Geometer: Graph Few-Shot Class-Incremental Learning via Prototype Representation

With the tremendous expansion of graphs data, node classification shows ...

Please sign up or login with your details

Forgot password? Click here to reset