  Model Selection is a key part of many ecological studies, with Akaike's
Information Criterion the most commonly used technique. Typically, a number of
candidate models are defined a priori and ranked according to their expected
out-of-sample performance. Model selection, however, only assesses the relative
performance of the models and, as pointed out in a recent paper, a large
proportion of ecology papers that use model selection do not assess the
absolute fit of the `best' model. In this paper, it is argued that assessing
the absolute fit of the `best' model alone does not go far enough. This is
because a model that appears to perform well under model selection is also
likely to appear to perform well under measures of absolute fit, even when
there is no predictive value. A model selection permutation test is proposed
that assesses the probability that the model selection statistic of the `best'
model could have occurred by chance alone, whilst taking account of
dependencies between the models. It is argued that this test should always be
performed before formal model selection takes place. The test is demonstrated
on two real population modelling examples of ibex in northern Italy and wild
reindeer in Norway.
