  Bayesian Neural Networks (BNNs) offer probability distributions for model
parameters, enabling uncertainty quantification in predictions. However, they
often underperform compared to deterministic neural networks. Utilizing mutual
learning can effectively enhance the performance of peer BNNs. In this paper,
we propose a novel approach to improve BNNs performance through deep mutual
learning. The proposed approaches aim to increase diversity in both network
parameter distributions and feature distributions, promoting peer networks to
acquire distinct features that capture different characteristics of the input,
which enhances the effectiveness of mutual learning. Experimental results
demonstrate significant improvements in the classification accuracy, negative
log-likelihood, and expected calibration error when compared to traditional
mutual learning for BNNs.
