Assessing the Significance of Model Selection in Ecology

by   Edward Wheatcroft, et al.

Model Selection is a key part of many ecological studies, with Akaike's Information Criterion the most commonly used technique. Typically, a number of candidate models are defined a priori and ranked according to their expected out-of-sample performance. Model selection, however, only assesses the relative performance of the models and, as pointed out in a recent paper, a large proportion of ecology papers that use model selection do not assess the absolute fit of the `best' model. In this paper, it is argued that assessing the absolute fit of the `best' model alone does not go far enough. This is because a model that appears to perform well under model selection is also likely to appear to perform well under measures of absolute fit, even when there is no predictive value. A model selection permutation test is proposed that assesses the probability that the model selection statistic of the `best' model could have occurred by chance alone, whilst taking account of dependencies between the models. It is argued that this test should always be performed before formal model selection takes place. The test is demonstrated on two real population modelling examples of ibex in northern Italy and wild reindeer in Norway.


page 1

page 2

page 3

page 4


Multi-model mimicry for model selection according to generalised goodness-of-fit criteria

Selecting between candidate models is at the core of statistical practic...

Parsimony in Model Selection: Tools for Assessing Fit Propensity

Theories can be represented as statistical models for empirical testing....

Small Data, Big Decisions: Model Selection in the Small-Data Regime

Highly overparametrized neural networks can display curiously strong gen...

Robust model selection between population growth and multiple merger coalescents

We study the effect of biological confounders on the model selection pro...

The effect of biologically mediated decay rates on modelling soil carbon sequestration in agricultural settings

Microbial biomass carbon (MBC), a crucial soil labile carbon fraction, i...

A better measure of relative prediction accuracy for model selection and model estimation

Surveys show that the mean absolute percentage error (MAPE) is the most ...

Bayesian data selection

Insights into complex, high-dimensional data can be obtained by discover...

Please sign up or login with your details

Forgot password? Click here to reset