A Tandem Learning Rule for Efficient and Rapid Inference on Deep Spiking Neural Networks

07/02/2019
by   Jibin Wu, et al.
0

Emerging neuromorphic computing (NC) architectures have shown compelling energy efficiency in machine learning tasks using spiking neural networks (SNNs). However, due to the non-differentiable nature of spiking neuronal functions, the standard error back-propagation algorithm is not directly applicable to SNNs. In this work, we propose a tandem learning framework, that consists of a SNN and an Artificial Neural Network (ANN) that share weights. The ANN is an auxiliary structure that facilitates the error back-propagation for the training of the SNN. To this end, we consider the spike count as the discrete neural representation and design ANN neuronal activation function that can effectively approximate the spike count of the coupled SNN. The SNNs that are trained with the proposed tandem learning rule show competitive classification accuracies on the CIFAR-10 and ImageNet-2012 datasets with significantly reduced inference time and total synaptic operations over other state-of-the-art SNN implementations. The proposed tandem learning rule offers a novel solution to training efficient, low latency and high accuracy deep SNNs with low computing resources.

READ FULL TEXT
research
07/02/2019

A Hybrid Learning Rule for Efficient and Rapid Inference with Spiking Neural Networks

The emerging neuromorphic computing (NC) architectures have shown compel...
research
02/15/2019

Deep Spiking Neural Network with Spike Count based Learning Rule

Deep spiking neural networks (SNNs) support asynchronous event-driven co...
research
08/09/2022

A Time-to-first-spike Coding and Conversion Aware Training for Energy-Efficient Deep Spiking Neural Network Processor Design

In this paper, we present an energy-efficient SNN architecture, which ca...
research
07/02/2020

Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks

Spiking neural networks (SNNs) have shown clear advantages over traditio...
research
10/10/2022

Training Spiking Neural Networks with Local Tandem Learning

Spiking neural networks (SNNs) are shown to be more biologically plausib...
research
11/18/2020

Temporal Surrogate Back-propagation for Spiking Neural Networks

Spiking neural networks (SNN) are usually more energy-efficient as compa...
research
07/25/2021

H2Learn: High-Efficiency Learning Accelerator for High-Accuracy Spiking Neural Networks

Although spiking neural networks (SNNs) take benefits from the bio-plaus...

Please sign up or login with your details

Forgot password? Click here to reset