Direct Training via Backpropagation for Ultra-low Latency Spiking Neural Networks with Multi-threshold

by   Changqing Xu, et al.

Spiking neural networks (SNNs) can utilize spatio-temporal information and have a nature of energy efficiency which is a good alternative to deep neural networks(DNNs). The event-driven information processing makes SNNs can reduce the expensive computation of DNNs and save a lot of energy consumption. However, high training and inference latency is a limitation of the development of deeper SNNs. SNNs usually need tens or even hundreds of time steps during the training and inference process which causes not only the increase of latency but also the waste of energy consumption. To overcome this problem, we proposed a novel training method based on backpropagation (BP) for ultra-low latency(1-2 time steps) SNN with multi-threshold. In order to increase the information capacity of each spike, we introduce the multi-threshold Leaky Integrate and Fired (LIF) model. In our proposed training method, we proposed three approximated derivative for spike activity to solve the problem of the non-differentiable issue which cause difficulties for direct training SNNs based on BP. The experimental results show that our proposed method achieves an average accuracy of 99.56 CIFAR10, respectively with only 2 time steps. For the CIFAR10 dataset, our proposed method achieve 1.12 direct trained SNNs with fewer time steps.


Ultra-low Latency Spiking Neural Networks with Spatio-Temporal Compression and Synaptic Convolutional Block

Spiking neural networks (SNNs), as one of the brain-inspired models, has...

One Timestep is All You Need: Training Spiking Neural Networks with Ultra Low Latency

Spiking Neural Networks (SNNs) are energy efficient alternatives to comm...

Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks

Compared with artificial neural networks (ANNs), spiking neural networks...

Temporal Contrastive Learning for Spiking Neural Networks

Biologically inspired spiking neural networks (SNNs) have garnered consi...

Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratch

Spiking Neural Networks (SNNs) have recently emerged as an alternative t...

T2FSNN: Deep Spiking Neural Networks with Time-to-first-spike Coding

Spiking neural networks (SNNs) have gained considerable interest due to ...

Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks

Spiking neural networks (SNNs) are bio-inspired neural networks with asy...

Please sign up or login with your details

Forgot password? Click here to reset