  Although automatic speech recognition (ASR) task has gained remarkable
success by sequence-to-sequence models, there are two main mismatches between
its training and testing that might lead to performance degradation: 1) The
typically used cross-entropy criterion aims to maximize log-likelihood of the
training data, while the performance is evaluated by word error rate (WER), not
log-likelihood; 2) The teacher-forcing method leads to the dependence on ground
truth during training, which means that model has never been exposed to its own
prediction before testing. In this paper, we propose an optimization method
called self-critical sequence training (SCST) to make the training procedure
much closer to the testing phase. As a reinforcement learning (RL) based
method, SCST utilizes a customized reward function to associate the training
criterion and WER. Furthermore, it removes the reliance on teacher-forcing and
harmonizes the model with respect to its inference procedure. We conducted
experiments on both clean and noisy speech datasets, and the results show that
the proposed SCST respectively achieves 8.7% and 7.8% relative improvements
over the baseline in terms of WER.
