DeepAI
Log In Sign Up

Learning the Pseudoinverse Solution to Network Weights

07/13/2012
by   Jonathan Tapson, et al.
0

The last decade has seen the parallel emergence in computational neuroscience and machine learning of neural network structures which spread the input signal randomly to a higher dimensional space; perform a nonlinear activation; and then solve for a regression or classification output by means of a mathematical pseudoinverse operation. In the field of neuromorphic engineering, these methods are increasingly popular for synthesizing biologically plausible neural networks, but the "learning method" - computation of the pseudoinverse by singular value decomposition - is problematic both for biological plausibility and because it is not an online or an adaptive method. We present an online or incremental method of computing the pseudoinverse, which we argue is biologically plausible as a learning method, and which can be made adaptable for non-stationary data streams. The method is significantly more memory-efficient than the conventional computation of pseudoinverses by singular value decomposition.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/30/2014

Online and Adaptive Pseudoinverse Solutions for ELM Weights

The ELM method has become widely used for classification and regressions...
05/13/2020

A 28-nm Convolutional Neuromorphic Processor Enabling Online Learning with Spike-Based Retinas

In an attempt to follow biological information representation and organi...
10/05/2022

Convergence rates of the Kaczmarz-Tanabe method for linear systems

In this paper, we investigate the Kaczmarz-Tanabe method for exact and i...
06/28/2019

Background Subtraction using Adaptive Singular Value Decomposition

An important task when processing sensor data is to distinguish relevant...
10/08/2019

Research on the Concept of Liquid State Machine

Liquid State Machine (LSM) is a neural model with real time computations...
05/20/2021

On preconditioning the state formulation of incremental weak constraint 4D-Var

Using a high degree of parallelism is essential to perform data assimila...