DeepAI AI Chat
Log In Sign Up

BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning

by   Saeed Reza Kheradpisheh, et al.
Shahid Beheshti University

We recently proposed the S4NN algorithm, essentially an adaptation of backpropagation to multilayer spiking neural networks that use simple non-leaky integrate-and-fire neurons and a form of temporal coding known as time-to-first-spike coding. With this coding scheme, neurons fire at most once per stimulus, but the firing order carries information. Here, we introduce BS4NN, a modification of S4NN in which the synaptic weights are constrained to be binary (+1 or -1), in order to decrease memory and computation footprints. This was done using two sets of weights: firstly, real-valued weights, updated by gradient descent, and used in the backward pass of backpropagation, and secondly, their signs, used in the forward pass. Similar strategies have been used to train (non-spiking) binarized neural networks. The main difference is that BS4NN operates in the time domain: spikes are propagated sequentially, and different neurons may reach their threshold at different times, which increases computational power. We validated BS4NN on two popular benchmarks, MNIST and Fashion MNIST, and obtained state-of-the-art accuracies for this sort of networks (97.0 respect to real-valued weights (0.4 demonstrated that BS4NN outperforms a simple BNN with the same architectures on those two datasets (by 0.2 leverages the temporal dimension.


page 6

page 7

page 9


Spike time displacement based error backpropagation in convolutional spiking neural networks

We recently proposed the STiDi-BP algorithm, which avoids backward recur...

Spike-inspired Rank Coding for Fast and Accurate Recurrent Neural Networks

Biological spiking neural networks (SNNs) can temporally encode informat...

Mean Field Bayes Backpropagation: scalable training of multilayer neural networks with binary weights

Significant success has been reported recently using deep neural network...

Temporally Efficient Deep Learning with Spikes

The vast majority of natural sensory data is temporally redundant. Video...

A Time Encoding approach to training Spiking Neural Networks

While Spiking Neural Networks (SNNs) have been gaining in popularity, it...

Neural Nets via Forward State Transformation and Backward Loss Transformation

This article studies (multilayer perceptron) neural networks with an emp...