Enabling Spike-based Backpropagation in State-of-the-art Deep Neural Network Architectures

by   Chankyu Lee, et al.

Spiking Neural Networks (SNNs) has recently emerged as a prominent neural computing paradigm. However, the typical shallow spiking network architectures have limited capacity for expressing complex representations, while training a very deep spiking network have not been successful so far. Diverse methods have been proposed to get around this issue such as converting off-line trained deep Artificial Neural Networks (ANNs) to SNNs. However, ANN-to-SNN conversion scheme fails to capture the temporal dynamics of a spiking system. On the other hand, it is still a difficult problem to directly train deep SNNs using input spike events due to the discontinuous and non-differentiable nature of the spike signals. To overcome this problem, we propose using differentiable (but approximate) activation for Leaky Integrate-and-Fire (LIF) spiking neurons to train deep convolutional SNNs with input spike events using spike-based backpropagation algorithm. Our experiments show the effectiveness of the proposed spike-based learning strategy on state-of-the-art deep networks (VGG and Residual architectures) by achieving the best classification accuracies in MNIST, SVHN and CIFAR-10 datasets compared to other SNNs trained with spike-based learning. Moreover, we analyze sparse event-driven computations to demonstrate the efficacy of proposed SNN training method for inference operation in the spiking domain.


page 5

page 14


Training Deep Spiking Neural Networks using Backpropagation

Deep spiking neural networks (SNNs) hold great potential for improving t...

SLAYER: Spike Layer Error Reassignment in Time

Configuring deep Spiking Neural Networks (SNNs) is an exciting research ...

Spike Event Based Learning in Neural Networks

A scheme is derived for learning connectivity in spiking neural networks...

Constructing Accurate and Efficient Deep Spiking Neural Networks with Double-threshold and Augmented Schemes

Spiking neural networks (SNNs) are considered as a potential candidate t...

T-NGA: Temporal Network Grafting Algorithm for Learning to Process Spiking Audio Sensor Events

Spiking silicon cochlea sensors encode sound as an asynchronous stream o...

Hybrid Spiking Neural Network Fine-tuning for Hippocampus Segmentation

Over the past decade, artificial neural networks (ANNs) have made tremen...

Custom DNN using Reward Modulated Inverted STDP Learning for Temporal Pattern Recognition

Temporal spike recognition plays a crucial role in various domains, incl...

Please sign up or login with your details

Forgot password? Click here to reset