  Bayesian model averaging is a practical method for dealing with uncertainty
due to model specification. Use of this technique requires the estimation of
model probability weights. In this work, we revisit the derivation of
estimators for these model weights. Use of the Kullback-Leibler divergence as a
starting point leads naturally to a number of alternative information criteria
suitable for Bayesian model weight estimation. We explore three such criteria,
known to the statistics literature before, in detail: a Bayesian analogue of
the Akaike information criterion which we call the BAIC, the Bayesian
predictive information criterion (BPIC), and the posterior predictive
information criterion (PPIC). We compare the use of these information criteria
in numerical analysis problems common in lattice field theory calculations. We
find that the PPIC has the most appealing theoretical properties and can give
the best performance in terms of model-averaging uncertainty, particularly in
the presence of noisy data, while the BAIC is a simple and reliable
alternative.
