Continual Learning in Deep Neural Networks by Using Kalman Optimiser

05/20/2019
by   Honglin Li, et al.
0

Learning and adapting to new distributions or learning new tasks sequentially without forgetting the previously learned knowledge is a challenging phenomenon in continual learning models. Most of the conventional deep learning models are not capable of learning new tasks sequentially in one model without forgetting the previously learned ones. We address this issue by using a Kalman Optimiser. The Kalman Optimiser divides the neural network into two parts: the long-term and short-term memory units. The long-term memory unit is used to remember the learned tasks and the short-term memory unit is to adapt to the new task. We have evaluated our method on MNIST, CIFAR10, CIFAR100 datasets and compare our results with state-of-the-art baseline models. The results show that our approach enables the model to continually learn and adapt to the new changes without forgetting the previously learned tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/20/2019

Continual Learning in Deep Neural Networks by Using a Kalman Optimiser

Learning and adapting to new distributions or learning new tasks sequent...
research
06/07/2016

Active Long Term Memory Networks

Continual Learning in artificial neural networks suffers from interferen...
research
08/04/2020

Applying Incremental Deep Neural Networks-based Posture Recognition Model for Injury Risk Assessment in Construction

Monitoring awkward postures is a proactive prevention for Musculoskeleta...
research
05/20/2019

Continual Learning in Deep Neural Network by Using a Kalman Optimiser

Learning and adapting to new distributions or learning new tasks sequent...
research
03/29/2023

Futures Quantitative Investment with Heterogeneous Continual Graph Neural Network

It is a challenging problem to predict trends of futures prices with tra...
research
02/22/2022

Increasing Depth of Neural Networks for Life-long Learning

Increasing neural network depth is a well-known method for improving neu...
research
07/20/2022

Continual Variational Autoencoder Learning via Online Cooperative Memorization

Due to their inference, data representation and reconstruction propertie...

Please sign up or login with your details

Forgot password? Click here to reset