  Ensembles improve prediction performance and allow uncertainty quantification
by aggregating predictions from multiple models. In deep ensembling, the
individual models are usually black box neural networks, or recently, partially
interpretable semi-structured deep transformation models. However,
interpretability of the ensemble members is generally lost upon aggregation.
This is a crucial drawback of deep ensembles in high-stake decision fields, in
which interpretable models are desired. We propose a novel transformation
ensemble which aggregates probabilistic predictions with the guarantee to
preserve interpretability and yield uniformly better predictions than the
ensemble members on average. Transformation ensembles are tailored towards
interpretable deep transformation models but are applicable to a wider range of
probabilistic neural networks. In experiments on several publicly available
data sets, we demonstrate that transformation ensembles perform on par with
classical deep ensembles in terms of prediction performance, discrimination,
and calibration. In addition, we demonstrate how transformation ensembles
quantify both aleatoric and epistemic uncertainty, and produce minimax optimal
predictions under certain conditions.
