  In this paper, we study the post-hoc calibration of modern neural networks, a
problem that has drawn a lot of attention in recent years. Many calibration
methods of varying complexity have been proposed for the task, but there is no
consensus about how expressive these should be. We focus on the task of
confidence scaling, specifically on post-hoc methods that generalize
Temperature Scaling, we call these the Adaptive Temperature Scaling family. We
analyse expressive functions that improve calibration and propose interpretable
methods. We show that when there is plenty of data complex models like neural
networks yield better performance, but are prone to fail when the amount of
data is limited, a common situation in certain post-hoc calibration
applications like medical diagnosis. We study the functions that expressive
methods learn under ideal conditions and design simpler methods but with a
strong inductive bias towards these well-performing functions. Concretely, we
propose Entropy-based Temperature Scaling, a simple method that scales the
confidence of a prediction according to its entropy. Results show that our
method obtains state-of-the-art performance when compared to others and, unlike
complex models, it is robust against data scarcity. Moreover, our proposed
model enables a deeper interpretation of the calibration process.
