Class-incremental Learning using a Sequence of Partial Implicitly Regularized Classifiers

04/04/2021
by   Sobirdzhon Bobiev, et al.
3

In class-incremental learning, the objective is to learn a number of classes sequentially without having access to the whole training data. However, due to a problem known as catastrophic forgetting, neural networks suffer substantial performance drop in such settings. The problem is often approached by experience replay, a method which stores a limited number of samples to be replayed in future steps to reduce forgetting of the learned classes. When using a pretrained network as a feature extractor, we show that instead of training a single classifier incrementally, it is better to train a number of specialized classifiers which do not interfere with each other yet can cooperatively predict a single class. Our experiments on CIFAR100 dataset show that the proposed method improves the performance over SOTA by a large margin.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

02/27/2020

Brain-Inspired Model for Incremental Learning Using a Few Examples

Incremental learning attempts to develop a classifier which learns conti...
02/18/2021

Essentials for Class Incremental Learning

Contemporary neural networks are limited in their ability to learn from ...
06/21/2021

Incremental Deep Neural Network Learning using Classification Confidence Thresholding

Most modern neural networks for classification fail to take into account...
08/31/2020

Initial Classifier Weights Replay for Memoryless Class Incremental Learning

Incremental Learning (IL) is useful when artificial systems need to deal...
10/16/2020

Class-incremental Learning with Pre-allocated Fixed Classifiers

In class-incremental learning, a learning agent faces a stream of data w...
12/12/2021

Improving Vision Transformers for Incremental Learning

This paper studies using Vision Transformers (ViT) in class incremental ...
06/12/2021

Knowledge Consolidation based Class Incremental Online Learning with Limited Data

We propose a novel approach for class incremental online learning in a l...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.