Multi-Compartment Variational Online Learning for Spiking Neural Networks

07/23/2020
by   Hyeryung Jang, et al.
0

Spiking Neural Networks (SNNs) offer a novel computational paradigm that captures some of efficiency of biological brains for inference and learning via recursive processing and binary neural activations. Most existing training algorithms for SNNs assume spiking neuron models in which a neuron outputs individual spikes as a function of the dynamics of an internal state variable known as membrane potential. This paper explores a more general model in which each spiking neuron contains multiple compartments, each tracking the dynamics of a distinct membrane potential, while sharing the same synaptic weights across compartments. It is demonstrated that learning rules based on probabilistic generalized linear neural models can leverage the presence of multiple compartments through modern variational inference based on importance weighting or generalized expectation-maximization. The key idea is to use the neural compartments to sample multiple independent spiking signals from hidden neurons so as to obtain better statistical estimates of the likelihood training criterion. The derived online learning algorithms follow three-factor rules with global learning signals. Experimental results on a structured output memorization task and classification task with a standard neuromorphic data set demonstrate significant improvements in terms of accuracy and calibration with an increasing number of compartments.

READ FULL TEXT
research
02/05/2021

Multi-Sample Online Learning for Spiking Neural Networks based on Generalized Expectation Maximization

Spiking Neural Networks (SNNs) offer a novel computational paradigm that...
research
12/10/2018

Spiking Neural Networks: A Stochastic Signal Processing Perspective

Spiking Neural Networks (SNNs) are distributed systems whose computing e...
research
04/20/2020

VOWEL: A Local Online Learning Rule for Recurrent Networks of Probabilistic Spiking Winner-Take-All Circuits

Networks of spiking neurons and Winner-Take-All spiking circuits (WTA-SN...
research
02/15/2016

Training of spiking neural networks based on information theoretic costs

Spiking neural network is a type of artificial neural network in which n...
research
07/21/2023

Neuromorphic Online Learning for Spatiotemporal Patterns with a Forward-only Timeline

Spiking neural networks (SNNs) are bio-plausible computing models with h...
research
09/21/2018

Stochasticity from function - why the Bayesian brain may need no noise

An increasing body of evidence suggests that the trial-to-trial variabil...
research
08/28/2022

Scalable Nanophotonic-Electronic Spiking Neural Networks

Spiking neural networks (SNN) provide a new computational paradigm capab...

Please sign up or login with your details

Forgot password? Click here to reset