DeepAI AI Chat
Log In Sign Up

A duality connecting neural network and cosmological dynamics

by   Sven Krippendorf, et al.
Durham University

We demonstrate that the dynamics of neural networks trained with gradient descent and the dynamics of scalar fields in a flat, vacuum energy dominated Universe are structurally profoundly related. This duality provides the framework for synergies between these systems, to understand and explain neural network dynamics and new ways of simulating and describing early Universe models. Working in the continuous-time limit of neural networks, we analytically match the dynamics of the mean background and the dynamics of small perturbations around the mean field, highlighting potential differences in separate limits. We perform empirical tests of this analytic description and quantitatively show the dependence of the effective field theory parameters on hyperparameters of the neural network. As a result of this duality, the cosmological constant is matched inversely to the learning rate in the gradient descent update.


page 1

page 2

page 3

page 4


Convex Analysis of the Mean Field Langevin Dynamics

As an example of the nonlinear Fokker-Planck equation, the mean field La...

Mean-Field Analysis of Two-Layer Neural Networks: Global Optimality with Linear Convergence Rates

We consider optimizing two-layer neural networks in the mean-field regim...

Stochastic Modified Flows, Mean-Field Limits and Dynamics of Stochastic Gradient Descent

We propose new limiting dynamics for stochastic gradient descent in the ...

A Dynamical Central Limit Theorem for Shallow Neural Networks

Recent theoretical work has characterized the dynamics of wide shallow n...

Convergence dynamics of Generative Adversarial Networks: the dual metric flows

Fitting neural networks often resorts to stochastic (or similar) gradien...

Duality and Stability in Complex Multiagent State-Dependent Network Dynamics

Many of the current challenges in science and engineering are related to...

An analytic theory of shallow networks dynamics for hinge loss classification

Neural networks have been shown to perform incredibly well in classifica...