DeepAI
Log In Sign Up

Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks

03/12/2021
by   Bojian Yin, et al.
0

Inspired by more detailed modeling of biological neurons, Spiking neural networks (SNNs) have been investigated both as more biologically plausible and potentially more powerful models of neural computation, and also with the aim of extracting biological neurons' energy efficiency; the performance of such networks however has remained lacking compared to classical artificial neural networks (ANNs). Here, we demonstrate how a novel surrogate gradient combined with recurrent networks of tunable and adaptive spiking neurons yields state-of-the-art for SNNs on challenging benchmarks in the time-domain, like speech and gesture recognition. This also exceeds the performance of standard classical recurrent neural networks (RNNs) and approaches that of the best modern ANNs. As these SNNs exhibit sparse spiking, we show that they theoretically are one to three orders of magnitude more computationally efficient compared to RNNs with comparable performance. Together, this positions SNNs as an attractive solution for AI hardware implementations.

READ FULL TEXT
05/24/2020

Effective and Efficient Computation with Multiple-timescale Spiking Recurrent Neural Networks

The emergence of brain-inspired neuromorphic computing as a paradigm for...
12/01/2022

Surrogate Gradient Spiking Neural Networks as Encoders for Large Vocabulary Continuous Speech Recognition

Compared to conventional artificial neurons that produce dense and real-...
10/02/2020

FPGA Implementation of Simplified Spiking Neural Network

Spiking Neural Networks (SNN) are third-generation Artificial Neural Net...
12/12/2020

Low-Order Model of Biological Neural Networks

A biologically plausible low-order model (LOM) of biological neural netw...
12/29/2018

Training dynamically balanced excitatory-inhibitory networks

The construction of biologically plausible models of neural circuits is ...
10/13/2017

Efficient Computation in Adaptive Artificial Spiking Neural Networks

Artificial Neural Networks (ANNs) are bio-inspired models of neural comp...
01/25/2019

Biologically inspired alternatives to backpropagation through time for learning in recurrent neural nets

The way how recurrently connected networks of spiking neurons in the bra...