Spike-based Residual Blocks

02/08/2021
by   Wei Fang, et al.
0

Deep Spiking Neural Networks (SNNs) are harder to train than ANNs because of their discrete binary activation and spatio-temporal domain error back-propagation. Considering the huge success of ResNet in ANNs' deep learning, it is natural to attempt to use residual learning to train deep SNNs. Previous Spiking ResNet used a similar residual block to the standard block of ResNet, which we regard as inadequate for SNNs and which still causes the degradation problem. In this paper, we propose the spike-element-wise (SEW) residual block and prove that it can easily implement the residual learning. We evaluate our SEW ResNet on ImageNet. The experiment results show that the SEW ResNet can obtain higher performance by simply adding more layers, providing a simple method to train deep SNNs.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/15/2021

Advancing Residual Learning towards Powerful Deep Spiking Neural Networks

Despite the rapid progress of neuromorphic computing, inadequate capacit...
research
10/12/2022

Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks

Spiking neural networks (SNNs) are bio-inspired neural networks with asy...
research
12/09/2021

Advancing Deep Residual Learning by Solving the Crux of Degradation in Spiking Neural Networks

Despite the rapid progress of neuromorphic computing, the inadequate dep...
research
03/31/2022

A Temporal-oriented Broadcast ResNet for COVID-19 Detection

Detecting COVID-19 from audio signals, such as breathing and coughing, c...
research
10/13/2017

Residual Connections Encourage Iterative Inference

Residual networks (Resnets) have become a prominent architecture in deep...
research
10/09/2022

Galaxy Spin Classification I: Z-wise vs S-wise Spirals With Chirality Equivariant Residual Network

The angular momentum of galaxies (galaxy spin) contains rich information...
research
04/16/2019

On the Mathematical Understanding of ResNet with Feynman Path Integral

In this paper, we aim to understand Residual Network (ResNet) in a scien...

Please sign up or login with your details

Forgot password? Click here to reset