DeepAI AI Chat
Log In Sign Up

Multi-model mimicry for model selection according to generalised goodness-of-fit criteria

by   Lachlann McArthur, et al.

Selecting between candidate models is at the core of statistical practice. As statistical modelling techniques are rapidly evolving, the need for similar evolution in the ways to select between candidate models is increasing. With Akaike's Information Criterion (AIC) and Bayesian Information Criterion (BIC) not applicable for all sets of candidate models, and likelihood not the only criterion upon which one might seek to compare models, a flexible model selection technique called model mimicry has emerged. Using simulation to compare models based upon generalised goodness-of-fit measures, model mimicry is available for any pair of candidate models, so long as (1) the models are able to be fit to observed data, (2) new sets of data can be simulated under the models, and (3) a metric exists by which a dataset's goodness-of-fit to the model can be calculated. In this manuscript, a variation of model mimicry for the simultaneous comparison of multiple models (multi-model mimicry) is outlined, placed in its historical context, and its effectiveness is demonstrated for simple examples. Classification techniques for model selection using output from multi-model mimicry are also explored.


page 1

page 2

page 3

page 4


Assessing the Significance of Model Selection in Ecology

Model Selection is a key part of many ecological studies, with Akaike's ...

Three Approaches to Probability Model Selection

This paper compares three approaches to the problem of selecting among p...

Complementary Goodness of Fit Procedure for Crash Frequency Models

This paper presents a new procedure for evaluating the goodness of fit o...

Data Consistency Approach to Model Validation

In scientific inference problems, the underlying statistical modeling as...

Model Selection for Production System via Automated Online Experiments

A challenge that machine learning practitioners in the industry face is ...

FIESTA: Fast IdEntification of State-of-The-Art models using adaptive bandit algorithms

We present FIESTA, a model selection approach that significantly reduces...

Parsimony in Model Selection: Tools for Assessing Fit Propensity

Theories can be represented as statistical models for empirical testing....