A correlation game for unsupervised learning yields computational interpretations of Hebbian excitation, anti-Hebbian inhibition, and synapse elimination

04/03/2017
by   H. Sebastian Seung, et al.
0

Much has been learned about plasticity of biological synapses from empirical studies. Hebbian plasticity is driven by correlated activity of presynaptic and postsynaptic neurons. Synapses that converge onto the same neuron often behave as if they compete for a fixed resource; some survive the competition while others are eliminated. To provide computational interpretations of these aspects of synaptic plasticity, we formulate unsupervised learning as a zero-sum game between Hebbian excitation and anti-Hebbian inhibition in a neural network model. The game formalizes the intuition that Hebbian excitation tries to maximize correlations of neurons with their inputs, while anti-Hebbian inhibition tries to decorrelate neurons from each other. We further include a model of synaptic competition, which enables a neuron to eliminate all connections except those from its most strongly correlated inputs. Through empirical studies, we show that this facilitates the learning of sensory features that resemble parts of objects.

READ FULL TEXT

page 8

page 15

research
12/30/2018

Unsupervised learning by a nonlinear network with Hebbian excitatory and anti-Hebbian inhibitory neurons

This paper introduces a rate-based nonlinear neural network in which exc...
research
12/31/2018

Two "correlation games" for a nonlinear network with Hebbian excitatory neurons and anti-Hebbian inhibitory neurons

A companion paper introduces a nonlinear network with Hebbian excitatory...
research
02/22/2023

Unsupervised 3D Object Learning through Neuron Activity aware Plasticity

We present an unsupervised deep learning model for 3D object classificat...
research
11/11/2019

How data, synapses and neurons interact with each other: a variational principle marrying gradient ascent and message passing

Unsupervised learning requiring only raw data is not only a fundamental ...
research
02/11/2023

Synaptic Stripping: How Pruning Can Bring Dead Neurons Back To Life

Rectified Linear Units (ReLU) are the default choice for activation func...
research
10/23/2012

A Self-Organized Neural Comparator

Learning algorithms need generally the possibility to compare several st...
research
07/28/2015

SynapCountJ --- a Tool for Analyzing Synaptic Densities in Neurons

The quantification of synapses is instrumental to measure the evolution ...

Please sign up or login with your details

Forgot password? Click here to reset