Incremental Classifier Learning Based on PEDCC-Loss and Cosine Distance

06/11/2019
by   Qiuyu Zhu, et al.
0

The main purpose of incremental learning is to learn new knowledge while not forgetting the knowledge which have been learned before. At present, the main challenge in this area is the catastrophe forgetting, namely the network will lose their performance in the old tasks after training for new tasks. In this paper, we introduce an ensemble method of incremental classifier to alleviate this problem, which is based on the cosine distance between the output feature and the pre-defined center, and can let each task to be preserved in different networks. During training, we make use of PEDCC-Loss to train the CNN network. In the stage of testing, the prediction is determined by the cosine distance between the network latent features and pre-defined center. The experimental results on EMINST and CIFAR100 show that our method outperforms the recent LwF method, which use the knowledge distillation, and iCaRL method, which keep some old samples while training for new task. The method can achieve the goal of not forgetting old knowledge while training new classes, and solve the problem of catastrophic forgetting better.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/27/2020

Two-Level Residual Distillation based Triple Network for Incremental Object Detection

Modern object detection methods based on convolutional neural network su...
research
01/30/2018

Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence

We study the incremental learning problem for the classification task, a...
research
04/12/2019

Incremental multi-domain learning with network latent tensor factorization

The prominence of deep learning, large amount of annotated data and incr...
research
12/15/2020

Class-incremental Learning with Rectified Feature-Graph Preservation

In this paper, we address the problem of distillation-based class-increm...
research
06/30/2021

When Video Classification Meets Incremental Classes

With the rapid development of social media, tremendous videos with new c...
research
08/10/2023

Towards General and Fast Video Derain via Knowledge Distillation

As a common natural weather condition, rain can obscure video frames and...
research
07/03/2021

Split-and-Bridge: Adaptable Class Incremental Learning within a Single Neural Network

Continual learning has been a major problem in the deep learning communi...

Please sign up or login with your details

Forgot password? Click here to reset