Spiking Deep Residual Network

04/28/2018
by   Yangfan Hu, et al.
0

Recently, spiking neural network (SNN) has received significant attentions for its biological plausibility. SNN theoretically has at least the same computational power as traditional artificial neural networks (ANNs), and it has the potential to achieve revolutionary energy-efficiency. However, at current stage, it is still a big challenge to train a very deep SNN. In this paper, we propose an efficient approach to build a spiking version of deep residual network (ResNet), which represents the state-of-the-art convolutional neural networks (CNNs). We employ the idea of converting a trained ResNet to a network of spiking neurons named Spiking ResNet. To address the conversion problem, we propose a shortcut normalisation mechanism to appropriately scale continuous-valued activations to match firing rates in SNN, and a layer-wise error compensation approach to reduce the error caused by discretisation. Experimental results on MNIST, CIFAR-10, and CIFAR-100 demonstrate that the proposed Spiking ResNet yields the state-of-the-art performance of SNNs.

READ FULL TEXT

page 2

page 6

page 7

12/13/2021

Human-Level Control through Directly-Trained Deep Spiking Q-Networks

As the third-generation neural networks, Spiking Neural Networks (SNNs) ...
02/25/2020

RMP-SNNs: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Networks

Spiking Neural Networks (SNNs) have recently attracted significant resea...
02/08/2021

Spike-based Residual Blocks

Deep Spiking Neural Networks (SNNs) are harder to train than ANNs becaus...
12/15/2021

Advancing Residual Learning towards Powerful Deep Spiking Neural Networks

Despite the rapid progress of neuromorphic computing, inadequate capacit...
12/13/2016

Theory and Tools for the Conversion of Analog to Spiking Convolutional Neural Networks

Deep convolutional neural networks (CNNs) have shown great potential for...
10/03/2022

Efficient Spiking Transformer Enabled By Partial Information

Spiking neural networks (SNNs) have received substantial attention in re...
12/05/2016

Deep Pyramidal Residual Networks with Separated Stochastic Depth

On general object recognition, Deep Convolutional Neural Networks (DCNNs...