Minimum Description Length Revisited

08/21/2019
by   Peter Grünwald, et al.
0

This is an up-to-date introduction to and overview of the Minimum Description Length (MDL) Principle, a theory of inductive inference that can be applied to general problems in statistics, machine learning and pattern recognition. While MDL was originally based on data compression ideas, this introduction can be read without any knowledge thereof. It takes into account all major developments since 2007, the last time an extensive overview was written. These include new methods for model selection and averaging and hypothesis testing, as well as the first completely general definition of MDL estimators. Incorporating these developments, MDL can be seen as a powerful extension of both penalized likelihood and Bayesian approaches, in which penalization functions and prior distributions are replaced by more general luckiness functions, average-case methodology is replaced by a more robust worst-case approach, and in which methods classically viewed as highly distinct, such as AIC vs BIC and cross-validation vs Bayes can, to a large extent, be viewed from a unified perspective.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/29/2022

Introduction to minimum message length inference

The aim of this manuscript is to introduce the Bayesian minimum message ...
research
05/17/2020

Marginal likelihood computation for model selection and hypothesis testing: an extensive review

This is an up-to-date introduction to, and overview of, marginal likelih...
research
07/28/2020

The Minimum Description Length Principle for Pattern Mining: A Survey

This is about the Minimum Description Length (MDL) principle applied to ...
research
01/27/1999

Minimum Description Length Induction, Bayesianism, and Kolmogorov Complexity

The relationship between the Bayesian approach and the minimum descripti...
research
10/02/2021

Graph Compression with Application to Model Selection

Many multivariate data such as social and biological data exhibit comple...
research
01/30/2013

Minimum Encoding Approaches for Predictive Modeling

We analyze differences between two information-theoretically motivated a...
research
05/30/2009

A Minimum Description Length Approach to Multitask Feature Selection

Many regression problems involve not one but several response variables ...

Please sign up or login with your details

Forgot password? Click here to reset