How consistent is my model with the data? Information-Theoretic Model Check

12/07/2017
by   Andreas Svensson, et al.
0

The choice of model class is fundamental in statistical learning and system identification, no matter whether the class is derived from physical principles or is a generic black-box. We develop a method to evaluate the specified model class by assessing its capability of reproducing data that is similar to the observed data record. This model check is based on the information-theoretic properties of models viewed as data generators and is applicable to e.g. sequential data and nonlinear dynamical models. The method can be understood as a specific two-sided posterior predictive test. We apply the information-theoretic model check to both synthetic and real data and compare it with a classical whiteness test.

READ FULL TEXT
research
12/07/2017

Is My Model Flexible Enough? Information-Theoretic Model Check

The choice of model class is fundamental in statistical learning and sys...
research
10/04/2021

Information-theoretic generalization bounds for black-box learning algorithms

We derive information-theoretic generalization bounds for supervised lea...
research
10/11/2019

Generic Bounds on the Maximum Deviations in Sequential Prediction: An Information-Theoretic Analysis

In this paper, we derive generic bounds on the maximum deviations in pre...
research
11/02/2017

Fast Information-theoretic Bayesian Optimisation

Information-theoretic Bayesian optimisation techniques have demonstrated...
research
08/17/2018

Data Consistency Approach to Model Validation

In scientific inference problems, the underlying statistical modeling as...
research
11/04/2019

Improving Supervised Phase Identification Through the Theory of Information Losses

This paper considers the problem of Phase Identification in power distri...
research
06/07/2021

An Information-theoretic Approach to Distribution Shifts

Safely deploying machine learning models to the real world is often a ch...

Please sign up or login with your details

Forgot password? Click here to reset