Is Deep Learning an RG Flow?

06/12/2019
by   Ellen de Mello Koch, et al.
0

Although there has been a rapid development of practical applications, theoretical explanations of deep learning are in their infancy. A possible starting point suggests that deep learning performs a sophisticated coarse graining. Coarse graining is the foundation of the renormalization group (RG), which provides a systematic construction of the theory of large scales starting from an underlying microscopic theory. In this way RG can be interpreted as providing a mechanism to explain the emergence of large scale structure, which is directly relevant to deep learning. We pursue the possibility that RG may provide a useful framework within which to pursue a theoretical explanation of deep learning. A statistical mechanics model for a magnet, the Ising model, is used to train an unsupervised RBM. The patterns generated by the trained RBM are compared to the configurations generated through a RG treatment of the Ising model. We argue that correlation functions between hidden and visible neurons are capable of diagnosing RG-like coarse graining. Numerical experiments show the presence of RG-like patterns in correlators computed using the trained RBMs. The observables we consider are also able to exhibit important differences between RG and deep learning.

READ FULL TEXT

page 9

page 10

research
02/07/2020

Short sighted deep learning

A theory explaining how deep learning works is yet to be developed. Prev...
research
08/21/2023

A Deep Dive into the Connections Between the Renormalization Group and Deep Learning in the Ising Model

The renormalization group (RG) is an essential technique in statistical ...
research
10/14/2014

An exact mapping between the Variational Renormalization Group and Deep Learning

Deep learning is a broad set of techniques that uses multiple layers of ...
research
12/17/2021

Neurashed: A Phenomenological Model for Imitating Deep Learning Training

To advance deep learning methodologies in the next decade, a theoretical...
research
11/07/2022

Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy

In recent years, deep learning has infiltrated every field it has touche...
research
01/22/2018

Scale-invariant Feature Extraction of Neural Network and Renormalization Group Flow

Theoretical understanding of how deep neural network (DNN) extracts feat...
research
01/27/2018

Graphic displays of MLB pitching mechanics and its evolutions in PITCHf/x data

Systemic and idiosyncratic patterns in pitching mechanics of 24 top star...

Please sign up or login with your details

Forgot password? Click here to reset