DeepAI AI Chat
Log In Sign Up

OvA-INN: Continual Learning with Invertible Neural Networks

06/24/2020
by   G. Hocquet, et al.
CEA
Paris-Sud University
0

In the field of Continual Learning, the objective is to learn several tasks one after the other without access to the data from previous tasks. Several solutions have been proposed to tackle this problem but they usually assume that the user knows which of the tasks to perform at test time on a particular sample, or rely on small samples from previous data and most of them suffer of a substantial drop in accuracy when updated with batches of only one class at a time. In this article, we propose a new method, OvA-INN, which is able to learn one class at a time and without storing any of the previous data. To achieve this, for each class, we train a specific Invertible Neural Network to extract the relevant features to compute the likelihood on this class. At test time, we can predict the class of a sample by identifying the network which predicted the highest likelihood. With this method, we show that we can take advantage of pretrained models by stacking an Invertible Network on top of a feature extractor. This way, we are able to outperform state-of-the-art approaches that rely on features learning for the Continual Learning of MNIST and CIFAR-100 datasets. In our experiments, we reach 72 our model one class at a time.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/10/2019

Bayesian Optimized Continual Learning with Attention Mechanism

Though neural networks have achieved much progress in various applicatio...
10/10/2022

A Simple Baseline that Questions the Use of Pretrained-Models in Continual Learning

With the success of pretraining techniques in representation learning, a...
02/20/2022

Efficient Continual Learning Ensembles in Neural Network Subspaces

A growing body of research in continual learning focuses on the catastro...
02/09/2022

False Memory Formation in Continual Learners Through Imperceptible Backdoor Trigger

In this brief, we show that sequentially learning new information presen...
05/28/2018

Keep and Learn: Continual Learning by Constraining the Latent Space for Knowledge Preservation in Neural Networks

Data is one of the most important factors in machine learning. However, ...
08/29/2022

"Prompt-Gamma Neutron Activation Analysis (PGNAA)" Metal Spectral Classification using Deep Learning Method

There is a pressing market demand to minimize the test time of Prompt Ga...
05/31/2021

A study on the plasticity of neural networks

One aim shared by multiple settings, such as continual learning or trans...