Machine Learning as Statistical Data Assimilation

10/19/2017
by   H. D. I. Abarbanel, et al.
0

We identify a strong equivalence between neural network based machine learning (ML) methods and the formulation of statistical data assimilation (DA), known to be a problem in statistical physics. DA, as used widely in physical and biological sciences, systematically transfers information in observations to a model of the processes producing the observations. The correspondence is that layer label in the ML setting is the analog of time in the data assimilation setting. Utilizing aspects of this equivalence we discuss how to establish the global minimum of the cost functions in the ML context, using a variational annealing method from DA. This provides a design method for optimal networks for ML applications and may serve as the basis for understanding the success of "deep learning". Results from an ML example are presented. When the layer label is taken to be continuous, the Euler-Lagrange equation for the ML optimization problem is an ordinary differential equation, and we see that the problem being solved is a two point boundary value problem. The use of continuous layers is denoted "deepest learning". The Hamiltonian version provides a direct rationale for back propagation as a solution method for the canonical momentum; however, it suggests other solution methods are to be preferred.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/05/2017

Machine Learning, Deepest Learning: Statistical Data Assimilation Problems

We formulate a strong equivalence between machine learning, artificial i...
research
09/09/2020

Combining data assimilation and machine learning to infer unresolved scale parametrisation

In recent years, machine learning (ML) has been proposed to devise data-...
research
10/23/2020

Using machine learning to correct model error in data assimilation and forecast applications

The idea of using machine learning (ML) methods to reconstruct the dynam...
research
07/06/2019

Precision annealing Monte Carlo methods for statistical data assimilation and machine learning

In statistical data assimilation (SDA) and supervised machine learning (...
research
10/03/2022

On The Effects Of Data Normalisation For Domain Adaptation On EEG Data

In the Machine Learning (ML) literature, a well-known problem is the Dat...
research
05/15/2023

Neural Oscillators are Universal

Coupled oscillators are being increasingly used as the basis of machine ...
research
03/18/2023

Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review

Data Assimilation (DA) and Uncertainty quantification (UQ) are extensive...

Please sign up or login with your details

Forgot password? Click here to reset