  Deep Neural Networks (DNNs) have achieved remarkable success in a variety of
tasks, especially when it comes to prediction accuracy. However, in complex
real-world scenarios, particularly in safety-critical applications, high
accuracy alone is not enough. Reliable uncertainty estimates are crucial.
Modern DNNs, often trained with cross-entropy loss, tend to be overconfident,
especially with ambiguous samples. To improve uncertainty calibration, many
techniques have been developed, but they often compromise prediction accuracy.
To tackle this challenge, we propose the ``mutual-transport ensemble'' (MTE).
This approach introduces a co-trained auxiliary model and adaptively
regularizes the cross-entropy loss using Kullback-Leibler (KL) divergence
between the prediction distributions of the primary and auxiliary models. We
conducted extensive studies on various benchmarks to validate the effectiveness
of our method. The results show that MTE can simultaneously enhance both
accuracy and uncertainty calibration. For example, on the CIFAR-100 dataset,
our MTE method on ResNet34/50 achieved significant improvements compared to
previous state-of-the-art method, with absolute accuracy increases of
2.4%/3.7%, relative reductions in ECE of $42.3%/29.4%, and relative reductions
in classwise-ECE of 11.6%/15.3%.
