End-to-End Incremental Learning

07/25/2018
by   Francisco M. Castro, et al.
0

Although deep learning approaches have stood out in recent years due to their state-of-the-art results, they continue to suffer from catastrophic forgetting, a dramatic decrease in overall performance when training with new classes added incrementally. This is due to current neural network architectures requiring the entire dataset, consisting of all the samples from the old as well as the new classes, to update the model -a requirement that becomes easily unsustainable as the number of classes grows. We address this issue with our approach to learn deep neural networks incrementally, using new data and only a small exemplar set corresponding to samples from the old classes. This is based on a loss composed of a distillation measure to retain the knowledge acquired from the old classes, and a cross-entropy loss to learn the new classes. Our incremental training is achieved while keeping the entire framework end-to-end, i.e., learning the data representation and the classifier jointly, unlike recent methods with no such guarantees. We evaluate our method extensively on the CIFAR-100 and ImageNet (ILSVRC 2012) image classification datasets, and show state-of-the-art performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/23/2021

Balanced Softmax Cross-Entropy for Incremental Learning

Deep neural networks are prone to catastrophic forgetting when increment...
research
03/30/2020

Incremental Learning In Online Scenario

Modern deep learning approaches have achieved great success in many visi...
research
08/04/2023

Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learning

Deep Neural Network (DNN) has achieved great success on datasets of clos...
research
11/20/2018

Learning without Memorizing

Incremental learning (IL) is an important task aimed to increase the cap...
research
12/01/2022

GMM-IL: Image Classification using Incrementally Learnt, Independent Probabilistic Models for Small Sample Sizes

Current deep learning classifiers, carry out supervised learning and sto...
research
05/30/2019

Large Scale Incremental Learning

Modern machine learning suffers from catastrophic forgetting when learni...
research
11/21/2021

Accretionary Learning with Deep Neural Networks

One of the fundamental limitations of Deep Neural Networks (DNN) is its ...

Please sign up or login with your details

Forgot password? Click here to reset