Desire Backpropagation: A Lightweight Training Algorithm for Multi-Layer Spiking Neural Networks based on Spike-Timing-Dependent Plasticity

11/10/2022
by   Daniel Gerlinghoff, et al.
0

Spiking neural networks (SNN) are a viable alternative to conventional artificial neural networks when energy efficiency and computational complexity are of importance. A major advantage of SNNs is their binary information transfer through spike trains. The training of SNN has, however, been a challenge, since neuron models are non-differentiable and traditional gradient-based backpropagation algorithms cannot be applied directly. Furthermore, spike-timing-dependent plasticity (STDP), albeit being a spike-based learning rule, updates weights locally and does not optimize for the output error of the network. We present desire backpropagation, a method to derive the desired spike activity of neurons from the output error. The loss function can then be evaluated locally for every neuron. Incorporating the desire values into the STDP weight update leads to global error minimization and increasing classification accuracy. At the same time, the neuron dynamics and computational efficiency of STDP are maintained, making it a spike-based supervised learning rule. We trained three-layer networks to classify MNIST and Fashion-MNIST images and reached an accuracy of 98.41 respectively. Furthermore, we show that desire backpropagation is computationally less complex than backpropagation in traditional neural networks.

READ FULL TEXT
research
11/29/2022

Timing-Based Backpropagation in Spiking Neural Networks Without Single-Spike Restrictions

We propose a novel backpropagation algorithm for training spiking neural...
research
10/21/2019

S4NN: temporal backpropagation for spiking neural networks with one spike per neuron

We propose a new supervised learning rule for multilayer spiking neural ...
research
07/27/2020

Supervised Learning in Temporally-Coded Spiking Neural Networks with Approximate Backpropagation

In this work we propose a new supervised learning method for temporally-...
research
05/04/2020

Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation

Spiking Neural Networks (SNNs) operate with asynchronous discrete events...
research
10/29/2017

Training Probabilistic Spiking Neural Networks with First-to-spike Decoding

Third-generation neural networks, or Spiking Neural Networks (SNNs), aim...
research
07/20/2022

A Temporally and Spatially Local Spike-based Backpropagation Algorithm to Enable Training in Hardware

Spiking Neural Networks (SNNs) have emerged as a hardware efficient arch...
research
07/24/2023

Sparse-firing regularization methods for spiking neural networks with time-to-first spike coding

The training of multilayer spiking neural networks (SNNs) using the erro...

Please sign up or login with your details

Forgot password? Click here to reset