  With the rise of Deep Neural Networks, machine learning systems are nowadays
ubiquitous in a number of real-world applications, which bears the need for
highly reliable models. This requires a thorough look not only at the accuracy
of such systems, but also at their predictive uncertainty. Hence, we propose a
novel technique (with two different variations, named M-ATTA and V-ATTA) based
on test time augmentation, to improve the uncertainty calibration of deep
models for image classification. By leveraging na adaptive weighting system,
M/V-ATTA improves uncertainty calibration without affecting the model's
accuracy. The performance of these techniques is evaluated by considering
diverse metrics related to uncertainty calibration, demonstrating their
robustness. Empirical results, obtained on CIFAR-10, CIFAR-100, Aerial Image
Dataset, as well as in two different scenarios under distribution-shift,
indicate that the proposed methods outperform several state-of-the-art post-hoc
calibration techniques. Furthermore, the methods proposed also show
improvements in terms of predictive entropy on out-of-distribution samples.
Code for M/V-ATTA available at: https://github.com/pedrormconde/MV-ATTA
