  We explore the arguments for maximizing the `evidence' as an algorithm for
model selection. We show, using a new definition of model complexity which we
term `flexibility', that maximizing the evidence should appeal to both Bayesian
and Frequentist statisticians. This is due to flexibility's unique position in
the exact decomposition of log-evidence into log-fit minus flexibility. In the
Gaussian linear model, flexibility is asymptotically equal to the Bayesian
Information Criterion (BIC) penalty, but we caution against using BIC in place
of flexibility for model selection.
