Optimal input representation in neural systems at the edge of chaos

07/12/2021
by   Guillermo B. Morales, et al.
0

Shedding light onto how biological systems represent, process and store information in noisy environments is a key and challenging goal. A stimulating, though controversial, hypothesis poses that operating in dynamical regimes near the edge of a phase transition, i.e. at criticality or the "edge of chaos", can provide information-processing living systems with important operational advantages, creating, e.g., an optimal trade-off between robustness and flexibility. Here, we elaborate on a recent theoretical result, which establishes that the spectrum of covariance matrices of neural networks representing complex inputs in a robust way needs to decay as a power-law of the rank, with an exponent close to unity, a result that has been indeed experimentally verified in neurons of the mouse visual cortex. Aimed at understanding and mimicking these results, we construct an artificial neural network and train it to classify images. Remarkably, we find that the best performance in such a task is obtained when the network operates near the critical point, at which the eigenspectrum of the covariance matrix follows the very same statistics as actual neurons do. Thus, we conclude that operating near criticality can also have – besides the usually alleged virtues – the advantage of allowing for flexible, robust and efficient input representations.

READ FULL TEXT

page 3

page 5

research
09/11/2019

Optimal Machine Intelligence Near the Edge of Chaos

It has long been suggested that living systems, in particular the brain,...
research
04/16/2023

Tracy-Widom distribution for the edge eigenvalues of elliptical model

In this paper, we study the largest eigenvalues of sample covariance mat...
research
03/19/2022

Quantum Neural Networks – Computational Field Theory and Dynamics

To address Quantum Artificial Neural Networks as quantum dynamical compu...
research
12/08/2020

On 1/n neural representation and robustness

Understanding the nature of representation in neural networks is a goal ...
research
04/09/2022

Neural networks embrace learned diversity

Diversity conveys advantages in nature, yet homogeneous neurons typicall...
research
07/05/2017

Spectral Modes of Network Dynamics Reveal Increased Informational Complexity Near Criticality

What does the informational complexity of dynamical networked systems te...
research
09/23/2022

Estimating Model Error Covariances with Artificial Neural Networks

Methods to deal with systematic model errors are an increasingly importa...

Please sign up or login with your details

Forgot password? Click here to reset