Gradient Descent for Spiking Neural Networks

06/14/2017
by   Dongsung Huh, et al.
0

Much of studies on neural computation are based on network models of static neurons that produce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete pulses called spikes. Research in spike-based computation has been impeded by the lack of efficient supervised learning algorithm for spiking networks. Here, we present a gradient descent method for optimizing spiking network models by introducing a differentiable formulation of spiking networks and deriving the exact gradient calculation. For demonstration, we trained recurrent spiking networks on two dynamic tasks: one that requires optimizing fast ( millisecond) spike-based interactions for efficient encoding of information, and a delayed memory XOR task over extended duration ( second). The results show that our method indeed optimizes the spiking network dynamics on the time scale of individual spikes as well as behavioral time scales. In conclusion, our result offers a general purpose supervised learning algorithm for spiking neural networks, thus advancing further investigations on spike-based computation.

READ FULL TEXT
research
02/10/2012

Supervised Learning in Multilayer Spiking Neural Networks

The current article introduces a supervised learning algorithm for multi...
research
06/27/2016

Supervised learning based on temporal coding in spiking neural networks

Gradient descent training techniques are remarkably successful in traini...
research
11/09/2019

Action Recognition Using Supervised Spiking Neural Networks

Biological neurons use spikes to process and learn temporally dynamic in...
research
06/02/2021

A Differentiable Point Process with Its Application to Spiking Neural Networks

This paper is concerned about a learning algorithm for a probabilistic m...
research
09/16/2021

Deep Spiking Neural Networks with Resonate-and-Fire Neurons

In this work, we explore a new Spiking Neural Network (SNN) formulation ...
research
05/23/2023

Toward spike-based stochastic neural computing

Inspired by the highly irregular spiking activity of cortical neurons, s...
research
12/02/2022

Loss shaping enhances exact gradient learning with EventProp in Spiking Neural Networks

In a recent paper Wunderlich and Pehle introduced the EventProp algorith...

Please sign up or login with your details

Forgot password? Click here to reset