Two "correlation games" for a nonlinear network with Hebbian excitatory neurons and anti-Hebbian inhibitory neurons

1 Network model with disynaptic inhibition

The disynaptic inhibition network (Fig. 1, left) has the activity dynamics,

	$x_{i}$	$:= {[(1 - d t) x_{i} + d t λ_{i}^{- 1} (m \sum a = 1 W_{i a} u_{a} - r \sum α = 1 y_{α} A_{α i})]}_{i}^{+}$		(1)
	$y_{α}$	$= n \sum i = 1 A_{α i} x_{i}$		(2)

Here $d t$

is a step size parameter, which can be set at a small constant value or adjusted adaptively. The activation function

{[z]}^{+} = max {z, 0}

is half-wave rectification. After the activities converge to a steady state, update the connection matrices via

	$Δ W_{i a}$	$\propto x_{i} u_{a} - γ W_{i a} - κ \sum b W_{i b}$		(3)
	$Δ A_{α j}$	$\propto y_{α} x_{j} - (q^{2} - p^{2}) A_{α j} - p^{2} \sum i A_{α i}$		(4)

where $γ > 0$ , $κ > 0$ , and $q^{2} > p^{2}$ . After the updates (3) and (4), any negative elements of $W$ and $A$ are zeroed to maintain nonnegativity. The divisive factor $λ_{i} > 0$ in Eq. (1) is updated via

Δ λ_{i} \propto x_{i}^{2} - q^{2}

(5)

Intuitions behind the model definitions are explained in the companion paper (Seung, 2018). The goal of the present paper is show how the network can be interpreted as a method of solving a zero-sum game.

2 Correlation game between connections

2.1 Formulation as constrained optimization

The first normative principle concerns transformation of a sequence of input vectors

u (1), \dots, u (T)

into a sequence of output vectors

x (1), \dots, x (T)

. Both input and output are assumed nonnegative. Define the input matrix

U = [u (1), \dots, u (T)]

as the matrix containing input vectors

u (t)

as its columns. The element

U_{a t}

is the

a

th component of

u (t)

. Similarly, define the output matrix

X = [x (1), \dots, x (T)]

as containing output vectors

x (t)

as its columns. Define the output-input correlation matrix is

Its $i a$ element is the time average of $x_{i} u_{a}$ , or $⟨ x_{i} u_{a} ⟩$ . Similarly, define the output-output correlation matrix

Its $i j$ element is the time average of $x_{i} x_{j}$ , or $⟨ x_{i} x_{j} ⟩$

. Note that “correlation matrix” is used to mean second moment matrix rather than covariance matrix. In other words, the correlation matrix does not involve subtraction of mean values. This is natural for sparse nonnegative variables, but covariance matrices may be substituted in other settings.

Problem 1 (Constrained optimization).

Define the goal of unsupervised learning as the constrained optimization

max X \geq 0 Φ^{*} (\frac{X U^{⊤}}{T}) subject to % copositivity of D - \frac{X X^{⊤}}{T}

(6)

where $D$ is a fixed matrix and $Φ^{*}$ is a scalar-valued function that is assumed monotone nondecreasing as a function of every element of its matrix-valued argument.

Monotonicity is an important assumption because it allows us to interpret the objective of Eq. (6) as maximization of input-output correlations.

2.2 Copositivity vs. nonnegativity

Seung and Zung (2017) introduced the principle

max X \geq 0 Φ^{*} (\frac{X U^{⊤}}{T}) subject to % nonnegativity of D - \frac{X X^{⊤}}{T}

(7)

which differs from Eq. (6) only by the substitution of “nonnegativity” for “copositivity.” (Here nonnegativity of a matrix is defined to mean nonnegativity of all its elements.) While the formalisms here are valid for arbitrary $D$ , a convenient choice is to set diagonal elements of $D$ to be $q^{2}$ and off-diagonal elements of $D$ to be $p^{2},$

D_{i j} = {\begin{matrix} q^{2}, & i = j, p^{2}, & i \neq j \end{matrix}

(8)

If $p$ is much smaller than $q$ , the nonnegativity constraint $⟨ x_{i} x_{j} ⟩ \leq D_{i j}$ in Eq. (7) amounts to decorrelation.

A symmetric matrix $S$ is said to be copositive when $v^{⊤} S v \geq 0$ for every nonnegative vector $v \geq 0$ . This constraint is analogous to positive semidefiniteness but is more complex because it cannot be reduced to a single eigenvalue constraint. Hahnloser et al. (2003) give sufficient and necessary conditions for copositivity involving eigenvalues of submatrices.

Nonnegativity of $S$ is a sufficient condition for copositivity of $S$ , but it is not a necessary condition. In particular, copositivity of $S - X X^{⊤} / T$ does not require nonnegativity, so a solution of Problem 1 may have $⟨ x_{i} x_{j} ⟩ > D_{i j}$ for some $i$ and $j$ .

A necessary condition for copositivity of $S$ is nonnegativity of its diagonal elements, since $e_{i}^{⊤} S e_{i} < 0$ if $S_{i i} < 0$ where $e_{1}, \dots, e_{n}$ denotes the standard basis for $R^{n}$ . In particular copositivity of $S = D - X X^{⊤} / T$ requires that $⟨ x_{i}^{2} ⟩ \leq D_{i i}$ for all $i$ . These inequalities will be called “power constraints,” because they limit the power in the outputs.

If either of the diagonal elements $S_{i i}$ and $S_{j j}$ vanish, then a necessary condition for copositivity is nonnegativity of the off-diagonal element $S_{i j}$ . Therefore $⟨ x_{i} x_{j} ⟩$ may exceed $D_{i j}$ in a solution of Problem 1 only if the power constraints for $i$ and $j$ are not saturated.

2.3 Correlation game from Legendre-Lagrangian duality

The copositivity constraint in Eq. (6) can be enforced by introducing Lagrange multipliers $A$ and $Λ$ ,

max X \geq 0 min A, Λ \geq 0 {Φ^{*} (\frac{X U^{⊤}}{T}) + \frac{1}{2} Tr A (D - \frac{X X^{⊤}}{T}) A^{T} + \frac{1}{2} Tr Λ (D - \frac{X X^{⊤}}{T})}

(9)

The Lagrange multiplier $A$ is a nonnegative $r \times n$ matrix. The outer maximum must choose $X$ so that $D - X X^{⊤} / T$ is copositive because otherwise the minimum with respect to $A$ is $- \infty$ . The Lagrange multiplier $Λ = diag {λ_{1}, \dots, λ_{n}}$ is a nonnegative diagonal matrix. The outer maximum must choose $X$ so that the diagonal elements of $D - X X^{⊤} / T$ are nonnegative because otherwise the minimum with respect to $Λ$ is $- \infty$ .

As mentioned above, copositivity of $D - X X^{⊤} / T$ by itself already implies that the diagonal elements are nonnegative. It follows that the Lagrange multiplier $Λ$ is redundant for the primal problem, though it does affect the dual problem. Similarly, adding extra rows to the Lagrange multiplier $A$ does not change the primal problem. For enforcing the copositivity constraint, it would be sufficient for $A$ to be $1 \times n$ . However, making $r > 1$ does affect the dual problem.

Problem 2 (Game between cells and connections).

Switching the order of min and max in Eq. (1) yields the dual problem,

min A, Λ \geq 0 max X \geq 0 {Φ^{*} (\frac{X U^{⊤}}{T}) + \frac{1}{2} Tr (D - \frac{X X^{⊤}}{T}) (A^{⊤} A + Λ)}

(10)

This is an upper bound for Eq. (9) by the minimax inequality.

At this point, it is convenient to define the objective function $Φ^{*}$ as the convex conjugate (Legendre-Fenchel transform) of a function $Φ$ ,

Φ^{*} (C)

= max W \geq 0 {\sum i a W_{i a} C_{i a} - Φ (W)} .

(11)

The nonnegativity constraint on $W$ in Eq. (11) guarantees that $Φ^{*} (C)$ is monotone nondecreasing as a function of every element of $C$ . The function $Φ$ can be interpreted as a regularizer or prior for the weight matrix $W$ .

With Legendre duality, a maximization with respect to $W$ is implicit in Eq. (11). Switching the order of $W$ and $X$ maximizations yields the following equivalent problem.

Problem 3 (Game between connections).

The Lagrangian dual of the constrained optimization in Problem 1 is

min A, Λ \geq 0 max W \geq 0 R (W, A^{⊤} A + Λ)

(12)

with payoff function defined by

R (W, L) = max X \geq 0 {\frac{1}{T} Tr W^{⊤} X U^{⊤} - Φ (W) + \frac{1}{2} Tr (D - \frac{X X^{⊤}}{T}) L}

(13)

The min-max problem can be interpreted as a zero-sum game between $W$ on the one hand and $A$ and $Λ$ on the other.

Problem 3 is closely related to the correlation game previously introduced by Seung and Zung (2017),

min L \geq 0 max W \geq 0 R (W, L)

(14)

Problem 3 constrains $L = A^{⊤} A + Λ$ for some nonnegative $A$ and $Λ$ , so it is an upper bound for Eq. (14). This is the mathematical interpretation of choosing a parametrized form $A^{⊤} A + Λ$ for the Lagrange multiplier $L$ , as was done by Seung (2018).

The network model of Section 1 follows by setting

Φ (W) = \frac{γ}{2} \sum i a W_{i a}^{2} + \frac{κ}{2} \sum i {(\sum a W_{i a})}_{i a}^{2}

in Eq. (13) and applying online projected gradient ascent to perform the maximizations in Eq. (12) and online projected gradient descent to perform the minimizations. For a more general choice of $Φ (W)$ , Eq. (3) should be replaced by

Δ W_{i a} \propto x_{i} u_{a} - \frac{\partial Φ}{\partial W_{i a}}

3 Correlation game between cells

The second normative principle concerns transformation of a sequence of nonnegative input vectors $u (1), \dots, u (T)$ into two sequences of nonnegative output vectors $x (1), \dots, x (T)$ and $y (1), \dots, y (T)$ . Define the input matrix $U = [u (1), \dots, u (T)]$ and the two output matrices $X = [x (1), \dots, x (T)]$ and $Y = [y (1), \dots, y (T)]$ .

Problem 4 (Game between cells).

Define the goal of unsupervised learning as the zero-sum game between $X$ and $Y$

max X \geq 0 min Y \geq 0 {Φ^{*} (\frac{X U^{⊤}}{T}) - Ψ^{*} (\frac{Y X^{⊤}}{T}) + \frac{1}{2} Tr \frac{Y Y^{⊤}}{T}}

(15)

where $Φ^{*}$ and $Ψ^{*}$ are scalar-valued functions assumed monotone nondecreasing as a function of every element of their matrix-valued arguments.

Note that only nonnegativity constraints remain in Problem 4; the copositivity constraint of Problem 1 is completely hidden. This correlation game can be interpreted as follows. The $E$ cells would like to maximize $S - E$ correlations (make $Φ^{*}$ large) and minimize $E - I$ correlations (make $Ψ^{*}$ small). The $I$ cells would like to maximize $I - E$ correlations (make $Ψ^{*}$ large) and minimize power (make $Tr Y Y^{⊤} / T$ small). There is conflict between the $E$ and $I$ cells because $E$ cells would like to minimize $E - I$ correlations while $I$ cells would like to maximize them. The compromise is that $E$ cells incompletely decorrelate from each other.

Problem 4 is equivalent to Problem 1 given the definition of

Ψ^{*} (C) = max A \geq 0 {\sum i α A_{α i} C_{α i} - Ψ (A)}

(16)

as the Legendre transform of

Ψ (A)

= \frac{1}{2} Tr A D A^{⊤}

(17)

Proof.

Substituting the definitions of Eqs. (16) and (17) into Eq. (15) yields

max X \geq 0 min A, Y \geq 0 {Φ^{*} (\frac{X U^{⊤}}{T}) - \frac{1}{T} Tr A^{⊤} Y X^{⊤} + \frac{1}{2} Tr A D A^{⊤} + \frac{1}{2} Tr \frac{Y Y^{⊤}}{T}}

This is minimized when $Y = A X$ , attaining the value

max X \geq 0 min A \geq 0 {Φ^{*} (\frac{X U^{⊤}}{T}) - \frac{1}{2 T} Tr A^{⊤} A X X^{⊤} + \frac{1}{2} Tr A D A^{⊤}}

This is identical to Eq. (9), except for the omission of the Lagrange multiplier $Λ$ which, as mentioned previously, is redundant in the primal problem. ∎

4 Discussion

The second normative principle is also interesting because it can be generalized to include $E - E$ and $I - I$ connections. This will be the subject of future work.

Acknowledgments

The author is grateful for helpful discussions with J. Zung, C. Pehlevan and D. Chklovskii. The research was supported in part by the Intelligence Advanced Research Projects Activity (IARPA) via DoI/IBC contract number D16PC0005, and by the National Institutes of Health via U19 NS104648 and U01 NS090562.

References

Földiák [1990] Peter Földiák. Forming sparse representations by local anti-hebbian learning. Biological cybernetics, 64(2):165–170, 1990.
Hahnloser et al. [2003] Richard HR Hahnloser, H Sebastian Seung, and Jean-Jacques Slotine. Permitted and forbidden sets in symmetric threshold-linear networks. Neural computation, 15(3):621–638, 2003.
Pehlevan and Chklovskii [2015] Cengiz Pehlevan and Dmitri Chklovskii. A normative theory of adaptive dimensionality reduction in neural networks. In Advances in neural information processing systems, pages 2269–2277, 2015.
Pehlevan et al. [2018] Cengiz Pehlevan, Anirvan M Sengupta, and Dmitri B Chklovskii. Why do similarity matching objectives lead to hebbian/anti-hebbian networks? Neural computation, 30(1):84–124, 2018.
Seung [2018] H Sebastian Seung. Unsupervised learning by a nonlinear network with hebbian excitatory and anti-hebbian inhibitory neurons. arXiv, 2018.
Seung and Zung [2017] H Sebastian Seung and Jonathan Zung. A correlation game for unsupervised learning yields computational interpretations of hebbian excitation, anti-hebbian inhibition, and synapse elimination. arXiv preprint arXiv:1704.00646, 2017.
Zylberberg et al. [2011] Joel Zylberberg, Jason Timothy Murphy, and Michael Robert DeWeese. A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of v1 simple cell receptive fields. PLoS Comput Biol, 7(10):e1002250, 2011.

Two "correlation games" for a nonlinear network with Hebbian excitatory neurons and anti-Hebbian inhibitory neurons

A correlation game for unsupervised learning yields computational interpretations of Hebbian excitation, anti-Hebbian inhibition, and synapse elimination

Unsupervised learning by a nonlinear network with Hebbian excitatory and anti-Hebbian inhibitory neurons

Linear Classification of Neural Manifolds with Correlated Variability

Metabolize Neural Network

Continual Learning with Deep Artificial Neurons

Unsupervised Learning through Temporal Smoothing and Entropy Maximization

Logical Information Cells I

1 Network model with disynaptic inhibition

2 Correlation game between connections

2.1 Formulation as constrained optimization

Problem 1 (Constrained optimization).

2.2 Copositivity vs. nonnegativity

2.3 Correlation game from Legendre-Lagrangian duality

Problem 2 (Game between cells and connections).

Problem 3 (Game between connections).

3 Correlation game between cells

Problem 4 (Game between cells).

Proof.

4 Discussion

Acknowledgments

References

Two "correlation games" for a nonlinear network with Hebbian excitatory neurons and anti-Hebbian inhibitory neurons

Related Research

A correlation game for unsupervised learning yields computational interpretations of Hebbian excitation, anti-Hebbian inhibition, and synapse elimination

Unsupervised learning by a nonlinear network with Hebbian excitatory and anti-Hebbian inhibitory neurons

Linear Classification of Neural Manifolds with Correlated Variability

Metabolize Neural Network

Continual Learning with Deep Artificial Neurons

Unsupervised Learning through Temporal Smoothing and Entropy Maximization

Logical Information Cells I

1 Network model with disynaptic inhibition

2 Correlation game between connections

2.1 Formulation as constrained optimization

Problem 1 (Constrained optimization).

2.2 Copositivity vs. nonnegativity

2.3 Correlation game from Legendre-Lagrangian duality

Problem 2 (Game between cells and connections).

Problem 3 (Game between connections).

3 Correlation game between cells

Problem 4 (Game between cells).

Proof.

4 Discussion

Acknowledgments

References