  This work introduces a novel probabilistic deep learning technique called
deep Gaussian mixture ensembles (DGMEs), which enables accurate quantification
of both epistemic and aleatoric uncertainty. By assuming the data generating
process follows that of a Gaussian mixture, DGMEs are capable of approximating
complex probability distributions, such as heavy-tailed or multimodal
distributions. Our contributions include the derivation of an
expectation-maximization (EM) algorithm used for learning the model parameters,
which results in an upper-bound on the log-likelihood of training data over
that of standard deep ensembles. Additionally, the proposed EM training
procedure allows for learning of mixture weights, which is not commonly done in
ensembles. Our experimental results demonstrate that DGMEs outperform
state-of-the-art uncertainty quantifying deep learning models in handling
complex predictive densities.
