  Despite their incredible performance, it is well reported that deep neural
networks tend to be overoptimistic about their prediction confidence. Finding
effective and efficient calibration methods for neural networks is therefore an
important endeavour towards better uncertainty quantification in deep learning.
In this manuscript, we introduce a novel calibration technique named
expectation consistency (EC), consisting of a post-training rescaling of the
last layer weights by enforcing that the average validation confidence
coincides with the average proportion of correct labels. First, we show that
the EC method achieves similar calibration performance to temperature scaling
(TS) across different neural network architectures and data sets, all while
requiring similar validation samples and computational resources. However, we
argue that EC provides a principled method grounded on a Bayesian optimality
principle known as the Nishimori identity. Next, we provide an asymptotic
characterization of both TS and EC in a synthetic setting and show that their
performance crucially depends on the target function. In particular, we discuss
examples where EC significantly outperforms TS.
