  Bayesian model averaging (BMA) is an approach to average over alternative
models; yet, it usually gets excessively concentrated around the single most
probable model, therefore achieving only sub-optimal classification
performance. The compression-based approach (Boulle, 2007) overcomes this
problem, averaging over the different models by applying a logarithmic
smoothing over the models' posterior probabilities. This approach has shown
excellent performances when applied to ensembles of naive Bayes classifiers.
AODE is another ensemble of models with high performance (Webb, 2005), based on
a collection of non-naive classifiers (called SPODE) whose probabilistic
predictions are aggregated by simple arithmetic mean. Aggregating the SPODEs
via BMA rather than by arithmetic mean deteriorates the performance; instead,
we aggregate the SPODEs via the compression coefficients and we show that the
resulting classifier obtains a slight but consistent improvement over AODE.
However, an important issue in any Bayesian ensemble of models is the
arbitrariness in the choice of the prior over the models. We address this
problem by the paradigm of credal classification, namely by substituting the
unique prior with a set of priors. Credal classifier automatically recognize
the prior-dependent instances, namely the instances whose most probable class
varies, when different priors are considered; in these cases, credal
classifiers remain reliable by returning a set of classes rather than a single
class. We thus develop the credal version of both the BMA-based and the
compression-based ensemble of SPODEs, substituting the single prior over the
models by a set of priors. Experiments show that both credal classifiers
provide higher classification reliability than their determinate counterparts;
moreover the compression-based credal classifier compares favorably to previous
credal classifiers.
