  Despite the success of deep neural network (DNN) on sequential data (i.e.,
scene text and speech) recognition, it suffers from the over-confidence problem
mainly due to overfitting in training with the cross-entropy loss, which may
make the decision-making less reliable. Confidence calibration has been
recently proposed as one effective solution to this problem. Nevertheless, the
majority of existing confidence calibration methods aims at non-sequential
data, which is limited if directly applied to sequential data since the
intrinsic contextual dependency in sequences or the class-specific statistical
prior is seldom exploited. To the end, we propose a Context-Aware Selective
Label Smoothing (CASLS) method for calibrating sequential data. The proposed
CASLS fully leverages the contextual dependency in sequences to construct
confusion matrices of contextual prediction statistics over different classes.
Class-specific error rates are then used to adjust the weights of smoothing
strength in order to achieve adaptive calibration. Experimental results on
sequence recognition tasks, including scene text recognition and speech
recognition, demonstrate that our method can achieve the state-of-the-art
performance.
