The Hidden-Manifold Hopfield Model and a learning phase transition

by   Matteo Negri, et al.
Sapienza University of Rome

The Hopfield model has a long-standing tradition in statistical physics, being one of the few neural networks for which a theory is available. Extending the theory of Hopfield models for correlated data could help understand the success of deep neural networks, for instance describing how they extract features from data. Motivated by this, we propose and investigate a generalized Hopfield model that we name Hidden-Manifold Hopfield Model: we generate the couplings from P=α N examples with the Hebb rule using a non-linear transformation of D=α_D N random vectors that we call factors, with N the number of neurons. Using the replica method, we obtain a phase diagram for the model that shows a phase transition where the factors hidden in the examples become attractors of the dynamics; this phase exists above a critical value of α and below a critical value of α_D. We call this behaviour learning transition.


Exact Phase Transitions in Deep Learning

This work reports deep-learning-unique first-order and second-order phas...

Detecting phase transitions in collective behavior using manifold's curvature

If a given behavior of a multi-agent system restricts the phase variable...

Generalisation error in learning with random features and the hidden manifold model

We study generalised linear regression and classification for a syntheti...

Absorbing Phase Transitions in Artificial Deep Neural Networks

Theoretical understanding of the behavior of infinitely-wide neural netw...

Phase transitions in semisupervised clustering of sparse networks

Predicting labels of nodes in a network, such as community memberships o...

Approximability of the Six-vertex Model

In this paper we take the first step toward a classification of the appr...

Simulating first-order phase transition with hierarchical autoregressive networks

We apply the Hierarchical Autoregressive Neural (HAN) network sampling a...

Please sign up or login with your details

Forgot password? Click here to reset