On the Algorithmic Power of Spiking Neural Networks

03/28/2018
by   Chi-Ning Chou, et al.
0

Spiking Neural Networks (SNN) are mathematical models in neuroscience to describe the dynamics among a set of neurons which interact with each other by firing spike signals to each other. Interestingly, recent works observed that for an integrate-and-fire model, when configured appropriately (e.g., after the parameters are learned properly), the neurons' firing rate, i.e., converges to an optimal solution of Lasso and certain quadratic optimization problems. Thus, SNN can be viewed as a natural algorithm for solving such convex optimization problems. However, theoretical understanding of SNN algorithms remains limited. In particular, only the convergence result for the Lasso problem is known, but the bounds of the convergence rate remain unknown. Therefore, we do not know any explicit complexity bounds for SNN algorithms. In this work, we investigate the algorithmic power of the integrate-and-fire SNN model after the parameters are properly learned/configured. In particular, we explore what algorithms SNN can implement. We start by formulating a clean discrete-time SNN model to facilitate the algorithmic study. We consider two SNN dynamics and obtain the following results. * We first consider an arguably simplest SNN dynamics with a threshold spiking rule, which we call simple SNN. We show that simple SNN solves the least square problem for a matrix A∈R^m× n and vector b∈R^m with timestep complexity O(κ n/ϵ). * For the under-determined case, we observe that simple SNN may solve the ℓ_1 minimization problem using an interesting primal-dual algorithm, which solves the dual problem by a gradient-based algorithm while updates the primal solution along the way. We analyze a variant dynamics and use simulation to serve as partial evidence to support the conjecture.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/15/2017

Sparse Coding by Spiking Neural Networks: Convergence Theory and Computational Results

In a spiking neural network (SNN), individual neurons operate autonomous...
research
04/25/2019

Winner-Take-All Computation in Spiking Neural Networks

In this work we study biological neural networks from an algorithmic per...
research
10/23/2018

Training Multilayer Spiking Neural Networks using NormAD based Spatio-Temporal Error Backpropagation

Spiking neural networks (SNNs) have garnered a great amount of interest ...
research
09/23/2015

Designing Behaviour in Bio-inspired Robots Using Associative Topologies of Spiking-Neural-Networks

This study explores the design and control of the behaviour of agents an...
research
03/30/2020

Genetic Algorithmic Parameter Optimisation of a Recurrent Spiking Neural Network Model

Neural networks are complex algorithms that loosely model the behaviour ...
research
06/21/2022

Structural Stability of Spiking Neural Networks

The past decades have witnessed an increasing interest in spiking neural...
research
12/09/2021

Continuation Path with Linear Convergence Rate

Path-following algorithms are frequently used in composite optimization ...

Please sign up or login with your details

Forgot password? Click here to reset