ScaIL: Classifier Weights Scaling for Class Incremental Learning

01/16/2020
by   Eden Belouadah, et al.
8

Incremental learning is useful if an AI agent needs to integrate data from a stream. The problem is non trivial if the agent runs on a limited computational budget and has a bounded memory of past data. In a deep learning approach, the constant computational budget requires the use of a fixed architecture for all incremental states. The bounded memory generates data imbalance in favor of new classes and a prediction bias toward them appears. This bias is commonly countered by introducing a data balancing step in addition to the basic network training. We depart from this approach and propose simple but efficient scaling of past class classifier weights to make them more comparable to those of new classes. Scaling exploits incremental state level statistics and is applied to the classifiers learned in the initial state of classes in order to profit from all their available data. We also question the utility of the widely used distillation loss component of incremental learning algorithms by comparing it to vanilla fine tuning in presence of a bounded memory. Evaluation is done against competitive baselines using four public datasets. Results show that the classifier weights scaling and the removal of the distillation are both beneficial.

READ FULL TEXT

page 2

page 14

research
08/31/2020

Initial Classifier Weights Replay for Memoryless Class Incremental Learning

Incremental Learning (IL) is useful when artificial systems need to deal...
research
02/01/2022

A Comparative Study of Calibration Methods for Imbalanced Class Incremental Learning

Deep learning approaches are successful in a wide range of AI problems a...
research
03/31/2020

A Simple Class Decision Balancing for Incremental Learning

Class incremental learning (CIL) problem, in which a learning agent cont...
research
08/20/2018

DeeSIL: Deep-Shallow Incremental Learning

Incremental Learning (IL) is an interesting AI problem when the algorith...
research
11/20/2018

Learning without Memorizing

Incremental learning (IL) is an important task aimed to increase the cap...
research
10/16/2020

Class-incremental Learning with Pre-allocated Fixed Classifiers

In class-incremental learning, a learning agent faces a stream of data w...
research
01/14/2023

RMM: Reinforced Memory Management for Class-Incremental Learning

Class-Incremental Learning (CIL) [40] trains classifiers under a strict ...

Please sign up or login with your details

Forgot password? Click here to reset