SupportNet: solving catastrophic forgetting in class incremental learning with support data

06/08/2018
by   Yu Li, et al.
0

A plain well-trained deep learning model often does not have the ability to learn new knowledge without forgetting the previously learned knowledge, which is known as the catastrophic forgetting. Here we propose a novel method, SupportNet, to solve the catastrophic forgetting problem in class incremental learning scenario efficiently and effectively. SupportNet combines the strength of deep learning and support vector machine (SVM), where SVM is used to identify the support data from the old data, which are fed to the deep learning model together with the new data for further training so that the model can review the essential information of the old data when learning the new information. Two powerful consolidation regularizers are applied to ensure the robustness of the learned model. Comprehensive experiments on various tasks, including enzyme function prediction, subcellular structure classification and breast tumor classification, show that SupportNet drastically outperforms the state-of-the-art incremental learning methods and even reaches similar performance as the deep learning model trained from scratch on both old and new data. Our program is accessible at: https://github.com/lykaust15/SupportNet

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/27/2020

Two-Level Residual Distillation based Triple Network for Incremental Object Detection

Modern object detection methods based on convolutional neural network su...
research
07/24/2018

Example Mining for Incremental Learning in Medical Imaging

Incremental Learning is well known machine learning approach wherein the...
research
11/30/2020

Incremental Learning via Rate Reduction

Current deep learning architectures suffer from catastrophic forgetting,...
research
08/27/2019

Learning Continually from Low-shot Data Stream

While deep learning has achieved remarkable results on various applicati...
research
05/04/2022

Explain to Not Forget: Defending Against Catastrophic Forgetting with XAI

The ability to continuously process and retain new information like we d...
research
10/05/2022

Multi-stream Fusion for Class Incremental Learning in Pill Image Classification

Classifying pill categories from real-world images is crucial for variou...
research
04/15/2022

Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection

Lifelong event detection aims to incrementally update a model with new e...

Please sign up or login with your details

Forgot password? Click here to reset