Towards a learning-theoretic analysis of spike-timing dependent plasticity

09/25/2012 ∙ by David Balduzzi, et al. ∙ Max Planck Society ETH Zurich 0

This paper suggests a learning-theoretic perspective on how synaptic plasticity benefits global brain functioning. We introduce a model, the selectron, that (i) arises as the fast time constant limit of leaky integrate-and-fire neurons equipped with spiking timing dependent plasticity (STDP) and (ii) is amenable to theoretical analysis. We show that the selectron encodes reward estimates into spikes and that an error bound on spikes is controlled by a spiking margin and the sum of synaptic weights. Moreover, the efficacy of spikes (their usefulness to other reward maximizing selectrons) also depends on total synaptic strength. Finally, based on our analysis, we propose a regularized version of STDP, and show the regularization improves the robustness of neuronal learning when faced with multiple stimuli.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 8

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Finding principles underlying learning in neural networks is an important problem for both artificial and biological networks. An elegant suggestion is that global objective functions may be optimized during learning

[1]. For biological networks however, the currently known neural plasticity mechanisms use a very restricted set of data – largely consisting of spikes and diffuse neuromodulatory signals. How a global optimization procedure could be implemented at the neuronal (cellular) level is thus a difficult problem.

A successful approach to this question has been Rosenblatt’s perceptron

[2]

and its extension to multilayer perceptrons via backpropagation

[3]

. Similarly, (restricted) Boltzmann machines, constructed from simple stochastic units, have provided a remarkably powerful approach to organizing distributed optimization across many layers

[4]. By contrast, although there has been significant progress in developing and understanding more biologically realistic models of neuronal learning [5, 6, 7, 8, 9, 10], these do not match the performance of simpler, more analytically and computationally tractable models in learning tasks.

Overview.

This paper constructs a bridge from biologically realistic to analytically tractable models. The selectron is a model derived from leaky integrate and fire neurons equipped with spike-timing dependent plasticity that is amenable to learning-theoretic analysis. Our aim is to extract some of the principles implicit in STDP by thoroughly investigating a limit case.

Section §2 introduces the selectron. We state a constrained reward maximization problem which implies that selectrons encode empirical reward estimates into spikes. Our first result, section §3, is that the selectron arises as the fast time constant limit of well-established models of neuronal spiking and plasticity, suggesting that cortical neurons may also be encoding reward estimates into their spiketrains.

Two important questions immediately arise. First, what guarantees can be provided on spikes being reliable predictors of global (neuromodulatory) outcomes? Second, what guarantees can be provided on the usefulness of spikes to other neurons? Sections §4 and §5 answer these questions by providing an upper bound on a suitably defined loss and a lower bound on the efficacy of a selectron’s spikes, measured in terms of its contribution to the expected reward of a downstream selectron. Both bounds are controlled by the sum of synaptic weights , thereby justifying the constraint introduced in §2. Finally, motivated by our analysis, §6 introduces a regularized STDP rule and shows that it learns more robustly than classical STDP. §7 concludes the paper. Proofs of theorems are provided in the supplementary material.

Related work.

Spike-timing dependent plasticity and its implications for the neural code have been intensively studied in recent years. The work closest in spirit to our own is Seung’s “hedonistic” synapses, which seek to increase average reward

[6]. Our work provides guarantees on the finite sample behavior of a discrete-time analog of hedonistic neurons. Another related line of research derives from the information bottleneck method [11, 9] which provides an alternate constraint to the one considered here. An information-theoretic perspective on synaptic homeostasis and metabolic cost, complementing the results in this paper, can be found in [12, 13]. Simulations combining synaptic renormalization with burst-STDP can be found in [14].

Important aspects of plasticity that we have not considered here are properties specific to continuous-time models, such as STDP’s behavior as a temporal filter [15], and also issues related to convergence [8, 10].

The learning-theoretic properties of neural networks have been intensively studied, mostly focusing on perceptrons, see for example [16]. A non-biologically motivated “large-margin” analog of the perceptron was proposed in [17].

2 The selectron

We introduce the selectron, which can be considered a biologically motivated adaptation of the perceptron, see §3. The mechanism governing whether or not the selectron spikes is a Heaviside function acting on a weighted sum of synaptic inputs; our contribution is to propose a new reward function and corresponding learning rule.

Let us establish some notation. Let denote the set of -dimensional

-valued vectors forming synaptic inputs to a selectron, and

the set of outputs. A selectron spikes according to

(1)

is the Heaviside function and is a valued -vector specifying the selectron’s synaptic weights. Let

denote the probability of input

arising.

To model the neuromodulatory system we introduce random variable

, where positive values correspond to desirable outcomes, negative to undesirable and zero to neutral. Let denote the probability of the release of neuromodulatory signal subsequent to input .

Definition 1.

Define reward function

(2)

The reward consists in three components. The first term is the neuromodulatory signal, which acts as a supervisor. The second term is the total current minus the threshold

. It is analogous to the margin in support vector machines or boosting algorithms, see section §

4 for a precise formulation.

The third term gates rewards according to whether or not the selectron spikes. The reward is thus selected111The name “selectron” was chosen to emphasize this selective aspect.: neuromodulatory signals are ignored by the selectron’s reward function when it does not spike, enabling specialization.

Constrained reward maximization.

The selectron solves the following optimization problem:

(3)
subject to:
Remark 1 (spikes encode rewards).


Optimization problem (3) ensures that selectrons spike for inputs that, on the basis of their empirical sample, reliably lead to neuromodulatory rewards. Thus, spikes encode expectations about rewards.

The constraint is motivated by the discussion after Theorem 1 and the analysis in §4 and §5. We postpone discussion of how to impose the constraint to §6, and focus on reward maximization here.

The reward maximization problem cannot be solved analytically in general. However, it is possible to use an iterative approach. Although is not continuous, the reward function is a continuous function of and is differentiable everywhere except for the “corner” where . We therefore apply gradient ascent by computing the derivative of (3) with respect to synaptic weights to obtain online learning rule

(4)

where update factor controls the learning rate.

The learning rule is selective: regardless of the neuromodulatory signal, synapse is updated only if there is both an input and output spike .

The selectron is not guaranteed to find a global optimum. It is prone to initial condition dependent local optima because rewards depend on output spikes in learning rule (4). Although this is an undesirable property for an isolated learner, it is less important, and perhaps even advantageous, in large populations where it encourages specialization.

Remark 2 (unsupervised setting).


Define the unsupervised setting by for all . The reward function reduces to . Without the constraint synapses will saturate. Imposing the constraint yields a more interesting solution where the selectron finds a weight vector summing to which balances (i) frequent spikes and (ii) high margins.

Theorem 1 (Controlling the frequency of spikes).


Assuming synaptic inputs are i.i.d. Bernoulli variables with , then

The Bernoulli regime is the discrete-time analog of the homogeneous Poisson setting used to prove convergence of reward-modulated STDP in [8]. Interestingly, in this setting the constraint provides a lever for controlling (lower bounding) rewards per spike

If inputs are not Bernoulli i.i.d., then and still covary, although the precise relationship is more difficult to quantify. Although i.i.d. inputs are unrealistic, note that recent neurophysiological evidence suggests neuronal firing – even of nearby neurons – is uncorrelated [18].

3 Relation to leaky integrate-and-fire neurons equipped with STDP

The literature contains an enormous variety of neuronal models, which vary dramatically in sophistication and the extent to which they incorporate the the details of the underlying biochemical processes. Similarly, there is a large menagerie of models of synaptic plasticity [19]. We consider two well-established models: Gerstner’s Spike Response Model (SRM) which generalizes leaky integrate-and-fire neurons [20] and the original spike-timing dependent plasticity learning rule proposed by Song et al [5], and show that the selectron arises in the fast time constant limit of the two models.

First let us recall the SRM. Suppose neuron last outputted a spike at time and receives input spikes at times from neuron . Neuron spikes or according to the Heaviside function applied to the membrane potential :

Input and output spikes add

to the membrane potential for and respectively. Here and are the membrane and synapse time constants.

The original STDP update rule [5] is

(5)

where and are time constants. STDP potentiates input synapses that spike prior to output spikes and depotentiates input synapses that spike subsequent to output spikes.

Theorem 2 (the selectron is the fast time constant limit of SRM + STDP).


In the fast time constant limit, , the SRM transforms into a selectron with

Moreover, STDP transforms into learning rule (4) in the unsupervised setting with for all . Finally, STDP arises as gradient ascent on a reward function whose limit is the unsupervised setting of reward function .

Theorem 2 shows that STDP implicitly maximizes a time-discounted analog of the reward function in (3). We expect many models of reward-modulated synaptic plasticity to be analytically tractable in the fast time constant limit. An important property shared by STDP and the selectron is that synaptic (de)potentiation is gated by output spikes, see §A.1 for a comparison with the perceptron which does not gate synaptic learning

4 An error bound

Maximizing reward function (3) implies that selectrons encode reward estimates into their spikes. Indeed, it recursively justifies incorporating spikes into the reward function via the margin , which only makes sense if upstream spikes predict reward. However, in a large system where estimates pile on top of each other there is a tendency to overfit, leading to poor generalizations [21]. It is therefore crucial to provide guarantees on the quality of spikes as estimators.

Boosting algorithms, where the outputs of many weak learners are aggregated into a classifier

[22], are remarkably resistant to overfitting as the number of learners increases [23]. Cortical learning may be analogous to boosting: individual neurons have access to a tiny fraction of the total brain state, and so are weak learners; and in the fast time constant limit, neurons are essentially aggregators.

We sharpen the analogy using the selectron. As a first step towards understanding how the cortex combats overfitting, we adapt a theorem developed to explain the effectiveness of boosting [24]. The goal is to show how the margin and constraint on synaptic weights improve error bounds.

Definition 2.

A selectron incurs a loss if a spike is followed by negative neuromodulatory feedback

(6)

The loss fails to take the estimates (spikes) of other selectrons into account and is difficult to optimize, so we also introduce the hinge loss:

(7)

Note that for all . Parameter controls the saturation point, beyond which the size of the margin makes no difference to .

An alternate loss222See §A.5 for an error bound. penalizes a selectron if it (i) fires when it shouldn’t, i.e. when or (ii) does not fire when it should, i.e. when . However, since the cortex contains many neurons and spiking is metabolically expensive [25], we propose a conservative loss that only penalizes errors of commission (“first, do no harm”) and does not penalize specialization.

Theorem 3 (spike error bound).


Suppose each selectron has synapses. For any selectron , let denote a 2-layer feedforward subnetwork. For all , with probability at least ,

Remark 3 (theoretical justification for maximizing margin and constraining ).


The theorem shows how subsets of distributed systems can avoid overfitting. First, it demonstrates the importance of maximizing the margin (i.e. the empirical reward). Second, it shows the capacity term depends on the number of synapses and the constraint on synaptic weights, rather than the capacity of – which can be very large.

The hinge loss is difficult to optimize directly since gating with output spikes renders it discontinuous. However, in the Bernoulli regime, Theorem 1 implies the bound in Theorem 3 can be rewritten as

(8)

and so again provides the lever required to control the loss. The constraint is best imposed offline, see §6.

5 A bound on the efficacy of inter-neuronal communication

Even if a neuron’s spikes perfectly predict positive neuromodulatory signals, the spikes only matter to the extent they affect other neurons in cortex. Spikes are produced for neurons by neurons. It is therefore crucial to provide guarantees on the usefulness of spikes.

In this section we quantify the effect of one selectron’s spikes on another selectron’s expected reward. We demonstrate a lower bound on efficacy and discuss its consequences.

Definition 3.

The efficacy of spikes from selectron on selectron is

i.e. the expected contribution of spikes from selectron to selectron ’s expected reward, relative to not spiking. The notation is intended to suggest an analogy with differentiation – the infinitesimal difference made by spikes on a single synapse.

Efficacy is zero if . In other words, if spikes from make no difference to the expected reward of .

The following theorem relies on the assumption that the average contribution of neuromodulators is higher after spikes than after it does not spike (i.e. upstream spikes predict reward), see §A.6 for precise statement. When the assumption is false the synapse should be pruned.

Theorem 4 (spike efficacy bound).


Let denote the frequency of spikes from neuron . The efficacy of ’s spikes on is lower bounded by

(9)

where is described in §A.6 and if and if .

The efficacy guarantee is interpreted as follows. First, the guarantee improves as co-spiking by and increases. However, the denominators imply that increasing the frequency of ’s spikes worsens the guarantee, insofar as is not correlated with . Similarly, from the third term, increasing ’s spikes worsens the guarantee if they do not correlate with .

An immediate corollary of Theorem 4 is that Hebbian learning rules, such as STDP and the selectron learning rule (4), improve the efficacy of spikes. However, it also shows that naively increasing the frequency of spikes carries a cost. Neurons therefore face a tradeoff. In fact, in the Bernoulli regime, Theorem 1 implies (9) can be rewritten as

(10)

so the constraint on synaptic strength can be used as a lever to improve guarantees on efficacy.

Remark 4 (efficacy improved by pruning weak synapses).


The term in (9) suggests that pruning weak synapses increases the efficacy of spikes, and so may aid learning in populations of selectrons or neurons.

6 Experiments

Cortical neurons are constantly exposed to different input patterns as organisms engage in different activities. It is therefore important that what neurons learn is robust to changing inputs [26, 27]. In this section, as proof of principle, we investigate a simple tweak of classical STDP involving offline regularization. We show that it improves robustness when neurons are exposed to more than one pattern.

Observe that regularizing optimization problem (3) yields

(11)
learning rule: (12)

incorporates synaptic renormalization directly into the update. However, (12) requires continuously re-evaluating the sum of synaptic weights. We therefore decouple learning into an online reward maximization phase and an offline regularization phase which resets the synaptic weights.

A similar decoupling may occur in cortex. It has recently been proposed that a function of NREM sleep may be to regulate synaptic weights [28]. Indeed, neurophysiological evidence suggests that average cortical firing rates increase during wakefulness and decrease during sleep, possibly reflecting synaptic strengths [29, 30]. Experimental evidence also points to a net increase in dendritic spines (synapses) during waking and a net decrease during sleep [31].

Setup.

We trained a neuron on a random input pattern for 10s to 87% accuracy with regularized STDP. See §A.7 for details on the structure of inputs. We then performed 700 trials (350 classical and 350 regularized) exposing the neuron to a new pattern for 20 seconds and observed performance under classical and regularized STDP.

SRM neurons with classical STDP.

We used Gerstner’s SRM model, recall §3, with parameters chosen to exactly coincide with [32]: , , , , and . STDP was implemented via (5) with parameters , , and also taken from [32]. Synaptic weights were clipped to fall in .

Regularized STDP

consists of a small tweak of classical STDP in the online phase, and an additional offline regularization phase:

  • Online. In the online phase, reduce the depotentiation bias from in the classical implementation to .

  • Offline. In the offline phase, modify synapses once per second according to

    (13)

    where is output spikes per second, is the target rate and update factor . The offline update rule is firing rate, and not spike, dependent.

Classical STDP has a depotentiation bias to prevent runaway potentiation feedback loops leading to seizures [5]. Since synapses are frequently renormalized offline we incorporate a weak exploratory (potentiation) bias during the online phase which helps avoid local minima.333The input stream contains a repeated pattern, so there is a potentiation bias in practice even though the net integral of STDP in the online phase is negative. This is in line with experimental evidence showing increased cortical activity during waking [30].

Since computing the sum of synaptic weights is non-physiological, we draw on Theorem 1 and use the neuron’s firing rate when responding to uncorrelated inputs as a proxy for . Thus, in the offline phase, synapses receive inputs generated as in the online phase but without repeated patterns. Note that (12) has a larger pruning effect on stronger synapses, discouraging specialization. Motivated by Remark 4, we introduce bias in the offline phase to ensure weaker synapses are downscaled more than strong synapses. For example, a synapse with is downscaled by twice as much as a synapse with weight .

Regularized STDP alternates between 2 seconds online and 4 seconds offline, which suffices to renormalize synaptic strengths. The frequency of the offline phase could be reduced by decreasing the update factors , presenting stimuli less frequently (than 7 times per second), or adding inhibitory neurons to the system.

Results.

A summary of results is presented in the table below: accuracy quantifies the fraction of spikes that co-occur with each pattern. Regularized STDP outperforms classical STDP on both patterns on average. It should be noted that regularized neurons were not only online for seconds but also offline – and exposed to Poisson noise – for seconds. Interestingly, exposure to Poisson noise improves performance.

Algorithm Accuracy
Pattern 1 Pattern 2
Classical 54% 39%
Regularized 59% 48%

Fig. 1 provides a more detailed analysis. Each panel shows a 2D-histogram (darker shades of gray correspond to more trials) plotting accuracies on both patterns simultaneously, and two 1D histograms plotting accuracies on the two patterns separately. The 1D histogram for regularized STDP shows a unimodal distribution for pattern #2, with most of the mass over accuracies of 50-90%. For pattern #1, which has been “unlearned” for twice as long as the training period, most of the mass is over accuracies of 50% to 90%, with a significant fraction “unlearnt”. By contrast, classical STDP exhibits extremely brittle behavior. It completely unlearns the original pattern in about half the trials, and also fails to learn the new pattern in most of the trials.

Thus, as suggested by our analysis, introducing a regularization both improves the robustness of STDP and enables an exploratory bias by preventing runaway feedback leading to epileptic seizures.

(a) Classical STDP
(b) Regularized STDP
Figure 1: Accuracy after 20 seconds of exposure to a novel pattern.

7 Discussion

The selectron provides a bridge between a particular model of spiking neurons – the Spike Response Model [20] with the original spike-timing dependent plasticity rule [5] – and models that are amenable to learning-theoretic analysis. Our hope is that the selectron and related models lead to an improved understanding of the principles underlying learning in cortex. It remains to be seen whether other STDP-based models also have tractable discrete-time analogs.

The selectron is an interesting model in its own right: it embeds reward estimates into spikes and maximizes a margin that improves error bounds. It imposes a constraint on synaptic weights that: concentrates rewards/spike, tightens error bounds and improves guarantees on spiking efficacy. Although the analysis does not apply directly to continuous-time models, experiments show that a tweak inspired by our analysis improves the performance of a more realistic model. An important avenue for future research is investigating the role of feedback in cortex, specifically NMDA synapses, which may have interesting learning-theoretic implications.

Acknowledgements. We thank Timothée Masquelier for generously sharing his source code [32] and Samory Kpotufe for useful discussions.

References

  • [1] Friston K, Kilner J, Harrison L: A free energy principle for the brain. J. Phys. Paris 2006, 100:70–87.
  • [2] Rosenblatt F: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 1958, 65(6):386–408.
  • [3] Rumelhart DE, Hinton GE, Williams RJ: Learning representations by back-propagating errors. Nature 1986, 323:533–536.
  • [4] Hinton G, Osindero S, Teh YW: A Fast Learning Algorithm for Deep Belief Nets. Neural Computation 2006, 18:1527–1554.
  • [5] Song S, Miller KD, Abbott LF: Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience 2000, 3(9).
  • [6] Seung HS: Learning in Spiking Neural Networks by Reinforcement of Stochastic Synaptic Transmission. Neuron 2003, 40(1063-1073).
  • [7] Bohte SM, Mozer MC: Reducing spike train variability: A computational theory of spike-timing dependent plasticity. In Advances in Neural Information Processing Systems (NIPS) 2005.
  • [8] Legenstein R, Maass W: A criterion for the convergence of learning with spike timing dependent plasticity. In Advances in Neural Information Processing Systems (NIPS) 2006.
  • [9] Buesing L, Maass W: Simplified rules and theoretical analysis for information bottleneck optimization and PCA with spiking neurons. In Adv in Neural Information Processing Systems (NIPS) 2007.
  • [10] Legenstein R, Pecevski D, Maass W: Theoretical analysis of learning with reward-modulated spike-timing-dependent plasticity. In Advances in Neural Information Processing Systems (NIPS) 2008.
  • [11] Tishby N, Pereira F, Bialek W: The information bottleneck method. In Proc. of the 37-th Annual Allerton Conference on Communication, Control and Computing. Edited by Hajek B, Sreenivas R 1999.
  • [12] Balduzzi D, Tononi G: What can neurons do for their brain? Communicate selectivity with spikes. To appear in Theory in Biosciences 2012.
  • [13] Balduzzi D, Ortega PA, Besserve M: Metabolic cost as an organizing principle for cooperative learning. Under review, missing 2012.
  • [14] Nere A, Olcese U, Balduzzi D, Tononi G: A neuromorphic architecture for object recognition and motion anticipation using burst-STDP. PLoS One 2012, 7(5):e36958.
  • [15] Schmiedt J, Albers C, Pawelzik K: Spike timing-dependent plasticity as dynamic filter. In Advances in Neural Information Processing Systems (NIPS) 2010.
  • [16] Anthony M, Bartlett PL: Neural Network Learning: Theoretical Foundations. Cambridge Univ Press 1999.
  • [17] Freund Y, Schapire RE: Large Margin Classification Using the Perceptron Algorithm. Machine Learning 1999, 37(3):277–296.
  • [18] Ecker AS, Berens P, Keliris GA, Bethge M, Logothetis NK, Tolias AS: Decorrelated neuronal firing in cortical microcircuits. Science 2010, 327(5965):584–7.
  • [19] Dan Y, Poo MM: Spike timing-dependent plasticity of neural circuits. Neuron 2004, 44:23–30.
  • [20] Gerstner W: Time structure of the activity in neural network models. Phys. Rev. E 1995, 51:738–758.
  • [21] Geman S, Bienenstock E, Doursat R:

    Neural Networks and the Bias/Variance Dilemma

    . Neural Comp 1992, 4:1–58.
  • [22] Freund Y, Schapire RE: Experiments with a New Boosting Algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference 1996.
  • [23] Schapire RE, Freund Y, Bartlett P, Lee WS: Boosting the Margin: A New Explanation for the Effectiveness of Voting Methods. The Annals of Statistics 1998, 26(5).
  • [24] Boucheron S, Bousquet O, Lugosi G: Theory of classification: A survey of some recent advances. ESAIM: PS 2005, 9:323–375.
  • [25] Hasenstaub A, Otte S, Callaway E, Sejnowski TJ: Metabolic cost as a unifying principle governing neuronal biophysics. Proc Natl Acad Sci U S A 2010, 107(27):12329–34.
  • [26] Fusi S, Drew P, Abbott L: Cascade Models of Synaptically Stored Memories. Neuron 2005, 45:599–611.
  • [27] Fusi S, Abbott L: Limits on the memory storage capacity of bounded synapses. Nature Neuroscience 2007, 10(4):485–493.
  • [28] Tononi G, Cirelli C: Sleep function and synaptic homeostasis. Sleep Med Rev 2006, 10:49–62.
  • [29] Vyazovskiy VV, Cirelli C, Pfister-Genskow M, Faraguna U, Tononi G: Molecular and electrophysiological evidence for net synaptic potentiation in wake and depression in sleep. Nat Neurosci 2008, 11(2):200–8.
  • [30] Vyazovskiy VV, Olcese U, Lazimy Y, Faraguna U, Esser SK, Williams JC, Cirelli C, Tononi G: Cortical firing and sleep homeostasis. Neuron 2009, 63(6):865–78.
  • [31] Maret S, Faraguna U, Nelson AB, Cirelli C, Tononi G: Sleep and waking modulate spine turnover in the adolescent mouse cortex. Nat Neurosci 2011, 14(11):1418–1420.
  • [32] Masquelier T, Thorpe SJ: Unsupervised learning of visual features through spike timing dependent plasticity. PLoS Comput Biol 2007, 3(2):e31.
  • [33] Roelfsema PR, van Ooyen A:

    Attention-gated reinforcement learning of internal representations for classification.

    Neural Comput 2005, 17(10):2176–2214.

a.1 The perceptron

We describe the perceptron to facilitate comparison with the selectron.

The perceptron’s loss function and learning rule are most naturally expressed when inputs and outputs take values in

. We therefore present the perceptron in both and “coordinate systems”.

Let us first relabel inputs and outputs from to : and . Given input , the perceptron’s output is determined according to

where is a real-valued -vector specifying the perceptron’s synaptic weights.

Given supervisor that labels inputs as belonging to one of two classes, define -valued loss function for the perceptron as

The following learning rule

converges onto an optimal solution if the classes are linearly separable.

The perceptron in “ coordinates”.

For the sake of comparison, we reformulate the perceptron in “ coordinates”. If for all , the mechanism of the perceptron is

(A.14)

Similarly, we obtain loss function

and learning rule

(A.15)

Non-biological features of the perceptron.

We highlight two features of the perceptron. First, learning rule (A.15) is not selective. If the perceptron classifies an input incorrectly, it updates its synaptic weights regardless of whether it outputted a 0 or a 1. It is thus not an output spike-dependent learning rule. The main consequence of this is that the perceptron is forced to classify every input. The selectron, by contrast, actively ignores neuromodulatory signals when it does not spike.

Second, the perceptron requires a local error signal. Multilayer perceptrons are constructed by replacing the in (A.14) with a differentiable function, such as the sigmoid, and backpropagating errors. However, backpropagation requires two pathways: one for feedforward spikes and another for feedback errors. In other words, the perceptron requires local error signals. A dedicated error pathway is biologically implausible [33], suggesting that the cortex relies on alternate mechanisms.

a.2 Proof of Theorem 1

The theorem requires computing the second moment of a selectron’s total current, given i.i.d. Bernoulli inputs. We also compute the expectation and the variance since these are of intrinsic interest.

Lemma 5 (moments for i.i.d. Bernoulli inputs).


For Bernoulli i.i.d. inputs on synapses, i.e. for all , we have

Proof.

For the mean,

since for all .

For the variance,

The expression for follows immediately. ∎

  • Assuming the inputs on each synapse are i.i.d. Bernoulli variables with , we have

Proof.

By Lemma 5, . Applying Chebyshev’s equality, , obtains

The result follows since . ∎

a.3 Proof of Theorem 2

We first compute the limit for STDP, and then move on to leaky integrate-and-fire neurons.

Lemma 6 (fast time constant limit of STDP).


The fast time constant limit of STDP, recall (5), is

where is the Dirac delta.

Proof.

We start from the STDP curve:

where is the Heaviside function. Note that we divide by to ensure that the area is constant as . An equivalent approach is to rescale and proportionately, so that both tend to zero.

Let us now compute the limit, in the sense of distributions, as . Let be a test function in the Schwartz space. The linear form associated to

is

Thus,

Integrating by parts obtains

so that

and . A similar argument holds for the negative part of the STDP curve, so the fast time constant limit is

Remark 5.

If STDP incorporates a potentiation bias (in the sense that the net integral is positive [5]), then the fast time constant limit acts exclusively by potentiation.

  • In the fast time constant limit, , the SRM transforms into a selectron with

    Moreover, STDP transforms into learning rule (4) in the unsupervised setting with for all . Finally, STDP arises as gradient ascent on a reward function whose limit is the unsupervised setting of reward function .

Proof.

Setting , , , and taking the limit yields

By Lemma 6, taking limits transforms STDP into

which is a special case of the online learning rule for the selectron derived in §2, where the neuromodulatory response is for all .

STDP is easily seen to arise as gradient ascent on

(A.16)

Taking the limit of (A.16) yields the special case of (2) where for all . ∎

Eq. (A.16) can be expressed in the shorthand

a.4 Proof of Theorem 3

To prove the theorem, we first recall an error bound from [24]. To state their result, we need the following notation. Let

where is a class of base classifiers taking values in .

Let be a nonnegative cost function such that , for example . Define

and (A.17)

and

and (A.18)
Theorem 7.

Let be a function chosen from based on data . With probability at least ,

(A.19)

where is the Lipschitz constant of , is a uniform upper bound on , and is the Rademacher complexity of on data .

Proof.

See §4.1 of [24]. ∎

We are interested in errors of commission, not omission, so we consider modified versions of (A.17),

and (A.20)

and (A.18),

and (A.21)

where we multiply by the indicator function . This results in a discontinuous hinge function. We therefore have to modify the above theorem.

Theorem 8.

Let be a function chosen from based on data . With probability at least ,

where is the Lipschitz constant of , is a uniform upper bound on , and is the Rademacher complexity of on data .

Proof.

We adapt the proof in [24]. By definition it follows that

Now observe that , where . Thus, changing distributions from to , we can write

where is the empirical distribution, and .

Continuing,

where we bound the two supremum’s with high probability separately using Rademacher complexity and apply the union bound. The last inequality follows since . ∎

Remark 6.

The change of distribution is not important, since the Rademacher complexity is bounded by the distribution free VC-dimension in Lemma 9.

To recover the setting of Theorem 3, we specialize to , for , so that the hinge loss can be written as . The Lipschitz constant is .

Now let

where functions in take values in .

Function class denotes functions implementable by a two-layer network of selectrons . The outputs of are aggregated, so that the function class of subnetwork is larger than that of selectrons considered individually, i.e. .

Lemma 9.

The Rademacher complexity of is upper bounded by

Proof.

Given selectron , let be the corresponding -valued function. Then

since for all . Thus,

The Rademacher complexity of is upper bounded by the VC-dimension,

which for a selectron with synapses is at most .

Finally, note that if then the selectron always spikes, and if then it never spikes. We therefore assume that , which implies and so