Surrogate Gradient Spiking Neural Networks as Encoders for Large Vocabulary Continuous Speech Recognition

by   Alexandre Bittar, et al.

Compared to conventional artificial neurons that produce dense and real-valued responses, biologically-inspired spiking neurons transmit sparse and binary information, which can also lead to energy-efficient implementations. Recent research has shown that spiking neural networks can be trained like standard recurrent neural networks using the surrogate gradient method. They have shown promising results on speech command recognition tasks. Using the same technique, we show that they are scalable to large vocabulary continuous speech recognition, where they are capable of replacing LSTMs in the encoder with only minor loss of performance. This suggests that they may be applicable to more involved sequence-to-sequence tasks. Moreover, in contrast to their recurrent non-spiking counterparts, they show robustness to exploding gradient problems without the need to use gates.


page 1

page 2

page 3

page 4


Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks

Inspired by more detailed modeling of biological neurons, Spiking neural...

Low-activity supervised convolutional spiking neural networks applied to speech commands recognition

Deep Neural Networks (DNNs) are the current state-of-the-art models in m...

Improving Surrogate Gradient Learning in Spiking Neural Networks via Regularization and Normalization

Spiking neural networks (SNNs) are different from the classical networks...

Deep Spiking Neural Networks for Large Vocabulary Automatic Speech Recognition

Artificial neural networks (ANN) have become the mainstream acoustic mod...

Spiking Neural Networks with Improved Inherent Recurrence Dynamics for Sequential Learning

Spiking neural networks (SNNs) with leaky integrate and fire (LIF) neuro...

Membrane Potential Distribution Adjustment and Parametric Surrogate Gradient in Spiking Neural Networks

As an emerging network model, spiking neural networks (SNNs) have arouse...

Spike-inspired Rank Coding for Fast and Accurate Recurrent Neural Networks

Biological spiking neural networks (SNNs) can temporally encode informat...

Please sign up or login with your details

Forgot password? Click here to reset