  Despite the impressive generalization capabilities of deep neural networks,
they have been repeatedly shown to be overconfident when they are wrong. Fixing
this issue is known as model calibration, and has consequently received much
attention in the form of modified training schemes and post-training
calibration procedures such as temperature scaling. While temperature scaling
is frequently used because of its simplicity, it is often outperformed by
modified training schemes. In this work, we identify a specific bottleneck for
the performance of temperature scaling. We show that for empirical risk
minimizers for a general set of distributions in which the supports of classes
have overlaps, the performance of temperature scaling degrades with the amount
of overlap between classes, and asymptotically becomes no better than random
when there are a large number of classes. On the other hand, we prove that
optimizing a modified form of the empirical risk induced by the Mixup data
augmentation technique can in fact lead to reasonably good calibration
performance, showing that training-time calibration may be necessary in some
situations. We also verify that our theoretical results reflect practice by
showing that Mixup significantly outperforms empirical risk minimization (with
respect to multiple calibration metrics) on image classification benchmarks
with class overlaps introduced in the form of label noise.
