  We discuss model selection to determine whether the variance-covariance
matrix of a multivariate Gaussian model with known mean should be considered to
be a constant diagonal, a non-constant diagonal, or an arbitrary positive
definite matrix. Of particular interest is the relationship between Bayesian
evidence and the flexibility penalty due to Priebe and Rougier. For the case of
an exponential family in canonical form equipped with a conjugate prior for the
canonical parameter, flexibility may be exactly decomposed into the usual BIC
likelihood penalty and a $O_p(1)$ term, the latter of which we explicitly
compute. We also investigate the asymptotics of Bayes factors for linearly
nested canonical exponential families equipped with conjugate priors; in
particular, we find the exact rates at which Bayes factors correctly diverge in
favor of the correct model: linearly and logarithmically in the number of
observations when the full and nested models are true, respectively. Such
theoretical considerations for the general case permit us to fully express the
asymptotic behavior of flexibility and Bayes factors for the
variance-covariance structure selection problem when we assume that the prior
for the model precision is a member of the gamma/Wishart family of
distributions or is uninformative. Simulations demonstrate evidence's immediate
and superior performance in model selection compared to approximate criteria
such as the BIC. We extend the framework to the multivariate Gaussian linear
model with three data-driven examples.
