  Generating confidence calibrated outputs is of utmost importance for the
applications of deep neural networks in safety-critical decision-making
systems. The output of a neural network is a probability distribution where the
scores are estimated confidences of the input belonging to the corresponding
classes, and hence they represent a complete estimate of the output likelihood
relative to all classes. In this paper, we propose a novel form of label
smoothing to improve confidence calibration. Since different classes are of
different intrinsic similarities, more similar classes should result in closer
probability values in the final output. This motivates the development of a new
smooth label where the label values are based on similarities with the
reference class. We adopt different similarity measurements, including those
that capture feature-based similarities or semantic similarity. We demonstrate
through extensive experiments, on various datasets and network architectures,
that our approach consistently outperforms state-of-the-art calibration
techniques including uniform label smoothing.
