Log In Sign Up

Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks

by   Bojian Yin, et al.

Inspired by more detailed modeling of biological neurons, Spiking neural networks (SNNs) have been investigated both as more biologically plausible and potentially more powerful models of neural computation, and also with the aim of extracting biological neurons' energy efficiency; the performance of such networks however has remained lacking compared to classical artificial neural networks (ANNs). Here, we demonstrate how a novel surrogate gradient combined with recurrent networks of tunable and adaptive spiking neurons yields state-of-the-art for SNNs on challenging benchmarks in the time-domain, like speech and gesture recognition. This also exceeds the performance of standard classical recurrent neural networks (RNNs) and approaches that of the best modern ANNs. As these SNNs exhibit sparse spiking, we show that they theoretically are one to three orders of magnitude more computationally efficient compared to RNNs with comparable performance. Together, this positions SNNs as an attractive solution for AI hardware implementations.


Effective and Efficient Computation with Multiple-timescale Spiking Recurrent Neural Networks

The emergence of brain-inspired neuromorphic computing as a paradigm for...

Surrogate Gradient Spiking Neural Networks as Encoders for Large Vocabulary Continuous Speech Recognition

Compared to conventional artificial neurons that produce dense and real-...

FPGA Implementation of Simplified Spiking Neural Network

Spiking Neural Networks (SNN) are third-generation Artificial Neural Net...

Low-Order Model of Biological Neural Networks

A biologically plausible low-order model (LOM) of biological neural netw...

Training dynamically balanced excitatory-inhibitory networks

The construction of biologically plausible models of neural circuits is ...

Efficient Computation in Adaptive Artificial Spiking Neural Networks

Artificial Neural Networks (ANNs) are bio-inspired models of neural comp...

Biologically inspired alternatives to backpropagation through time for learning in recurrent neural nets

The way how recurrently connected networks of spiking neurons in the bra...