SuperSpike: Supervised learning in multi-layer spiking neural networks

05/31/2017
by   Friedemann Zenke, et al.
0

A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in-vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in-silico. Here we revisit the problem of supervised learning in temporally coding multi-layer spiking neural networks. First, by using a surrogate gradient approach, we derive SuperSpike, a nonlinear voltage-based three factor learning rule capable of training multi-layer networks of deterministic integrate-and-fire neurons to perform nonlinear computations on spatiotemporal spike patterns. Second, inspired by recent results on feedback alignment, we compare the performance of our learning rule under different credit assignment strategies for propagating output errors to hidden units. Specifically, we test uniform, symmetric and random feedback, finding that simpler tasks can be solved with any type of feedback, while more complex tasks require symmetric feedback. In summary, our results open the door to obtaining a better scientific understanding of learning and computation in spiking neural networks by advancing our ability to train them to solve nonlinear problems involving transformations between different spatiotemporal spike-time patterns.

READ FULL TEXT
research
08/16/2023

Expressivity of Spiking Neural Networks

This article studies the expressive power of spiking neural networks whe...
research
11/12/2017

BP-STDP: Approximating Backpropagation using Spike Timing Dependent Plasticity

The problem of training spiking neural networks (SNNs) is a necessary pr...
research
03/30/2023

Learning Spiking Neural Systems with the Event-Driven Forward-Forward Process

We develop a novel credit assignment algorithm for information processin...
research
05/12/2020

Training spiking neural networks using reinforcement learning

Neurons in the brain communicate with each other through discrete action...
research
11/02/2020

Controlled Perturbation-Induced Switching in Pulse-Coupled Oscillator Networks

Pulse-coupled systems such as spiking neural networks exhibit nontrivial...
research
03/04/2019

Evolving Spiking Neural Networks for Nonlinear Control Problems

Spiking Neural Networks are powerful computational modelling tools that ...
research
10/13/2021

A Time Encoding approach to training Spiking Neural Networks

While Spiking Neural Networks (SNNs) have been gaining in popularity, it...

Please sign up or login with your details

Forgot password? Click here to reset