Continuous learning of spiking networks trained with local rules

11/18/2021
by   Dmitry Antonov, et al.
0

Artificial neural networks (ANNs) experience catastrophic forgetting (CF) during sequential learning. In contrast, the brain can learn continuously without any signs of catastrophic forgetting. Spiking neural networks (SNNs) are the next generation of ANNs with many features borrowed from biological neural networks. Thus, SNNs potentially promise better resilience to CF. In this paper, we study the susceptibility of SNNs to CF and test several biologically inspired methods for mitigating catastrophic forgetting. SNNs are trained with biologically plausible local training rules based on spike-timing-dependent plasticity (STDP). Local training prohibits the direct use of CF prevention methods based on gradients of a global loss function. We developed and tested the method to determine the importance of synapses (weights) based on stochastic Langevin dynamics without the need for the gradients. Several other methods of catastrophic forgetting prevention adapted from analog neural networks were tested as well. The experiments were performed on freely available datasets in the SpykeTorch environment.

READ FULL TEXT
research
03/28/2019

Deep Convolutional Spiking Neural Networks for Image Classification

Spiking neural networks are biologically plausible counterparts of the a...
research
08/01/2019

Biologically inspired sleep algorithm for artificial neural networks

Sleep plays an important role in incremental learning and consolidation ...
research
08/11/2022

Empirical investigations on WVA structural issues

In this paper we want to present the results of empirical verification o...
research
02/08/2019

Stimulating STDP to Exploit Locality for Lifelong Learning without Catastrophic Forgetting

Stochastic gradient descent requires that training samples be drawn from...
research
11/06/2018

On the role of neurogenesis in overcoming catastrophic forgetting

Lifelong learning capabilities are crucial for artificial autonomous age...
research
10/21/2022

Biologically Plausible Variational Policy Gradient with Spiking Recurrent Winner-Take-All Networks

One stream of reinforcement learning research is exploring biologically ...
research
03/22/2017

ASP: Learning to Forget with Adaptive Synaptic Plasticity in Spiking Neural Networks

A fundamental feature of learning in animals is the "ability to forget" ...

Please sign up or login with your details

Forgot password? Click here to reset