Learning to Time-Decode in Spiking Neural Networks Through the Information Bottleneck

06/02/2021
by   Nicolas Skatchkovsky, et al.
0

One of the key challenges in training Spiking Neural Networks (SNNs) is that target outputs typically come in the form of natural signals, such as labels for classification or images for generative models, and need to be encoded into spikes. This is done by handcrafting target spiking signals, which in turn implicitly fixes the mechanisms used to decode spikes into natural signals, e.g., rate decoding. The arbitrary choice of target signals and decoding rule generally impairs the capacity of the SNN to encode and process information in the timing of spikes. To address this problem, this work introduces a hybrid variational autoencoder architecture, consisting of an encoding SNN and a decoding Artificial Neural Network (ANN). The role of the decoding ANN is to learn how to best convert the spiking signals output by the SNN into the target natural signal. A novel end-to-end learning rule is introduced that optimizes a directed information bottleneck training criterion via surrogate gradients. We demonstrate the applicability of the technique in an experimental settings on various tasks, including real-life datasets.

READ FULL TEXT

page 8

page 9

page 14

research
01/28/2019

Surrogate Gradient Learning in Spiking Neural Networks

A growing number of neuromorphic spiking neural network processors that ...
research
11/10/2018

Efficient Spiking Neural Networks with Logarithmic Temporal Coding

A Spiking Neural Network (SNN) can be trained indirectly by first traini...
research
06/06/2017

Information Bottleneck in Control Tasks with Recurrent Spiking Neural Networks

The nervous system encodes continuous information from the environment i...
research
05/31/2022

Gluing Neural Networks Symbolically Through Hyperdimensional Computing

Hyperdimensional Computing affords simple, yet powerful operations to cr...
research
05/17/2023

Spiking Generative Adversarial Network with Attention Scoring Decoding

Generative models based on neural networks present a substantial challen...
research
08/28/2010

Spiking Neurons with ASNN Based-Methods for the Neural Block Cipher

Problem statement: This paper examines Artificial Spiking Neural Network...

Please sign up or login with your details

Forgot password? Click here to reset