Robust Modeling of Unknown Dynamical Systems via Ensemble Averaged Learning

03/07/2022
by   Victor Churchill, et al.
0

Recent work has focused on data-driven learning of the evolution of unknown systems via deep neural networks (DNNs), with the goal of conducting long time prediction of the evolution of the unknown system. Training a DNN with low generalization error is a particularly important task in this case as error is accumulated over time. Because of the inherent randomness in DNN training, chiefly in stochastic optimization, there is uncertainty in the resulting prediction, and therefore in the generalization error. Hence, the generalization error can be viewed as a random variable with some probability distribution. Well-trained DNNs, particularly those with many hyperparameters, typically result in probability distributions for generalization error with low bias but high variance. High variance causes variability and unpredictably in the results of a trained DNN. This paper presents a computational technique which decreases the variance of the generalization error, thereby improving the reliability of the DNN model to generalize consistently. In the proposed ensemble averaging method, multiple models are independently trained and model predictions are averaged at each time step. A mathematical foundation for the method is presented, including results regarding the distribution of the local truncation error. In addition, three time-dependent differential equation problems are considered as numerical examples, demonstrating the effectiveness of the method to decrease variance of DNN predictions generally.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/03/2022

Learning Fine Scale Dynamics from Coarse Observations via Inner Recurrence

Recent work has focused on data-driven learning of the evolution of unkn...
research
02/03/2022

Modeling unknown dynamical systems with hidden parameters

We present a data-driven numerical approach for modeling unknown dynamic...
research
07/20/2023

Flow Map Learning for Unknown Dynamical Systems: Overview, Implementation, and Benchmarks

Flow map learning (FML), in conjunction with deep neural networks (DNNs)...
research
09/28/2021

slimTrain – A Stochastic Approximation Method for Training Separable Deep Neural Networks

Deep neural networks (DNNs) have shown their success as high-dimensional...
research
02/24/2021

Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks

In order to safely deploy Deep Neural Networks (DNNs) within the percept...
research
02/07/2023

Deep-OSG: A deep learning approach for approximating a family of operators in semigroup to model unknown autonomous systems

This paper proposes a novel deep learning approach for approximating evo...
research
12/22/2022

A Mathematical Framework for Learning Probability Distributions

The modeling of probability distributions, specifically generative model...

Please sign up or login with your details

Forgot password? Click here to reset