  Sequence-to-sequence models, such as attention-based models in automatic
speech recognition (ASR), are typically trained to optimize the cross-entropy
criterion which corresponds to improving the log-likelihood of the data.
However, system performance is usually measured in terms of word error rate
(WER), not log-likelihood. Traditional ASR systems benefit from discriminative
sequence training which optimizes criteria such as the state-level minimum
Bayes risk (sMBR) which are more closely related to WER. In the present work,
we explore techniques to train attention-based models to directly minimize
expected word error rate. We consider two loss functions which approximate the
expected number of word errors: either by sampling from the model, or by using
N-best lists of decoded hypotheses, which we find to be more effective than the
sampling-based method. In experimental evaluations, we find that the proposed
training procedure improves performance by up to 8.2% relative to the baseline
system. This allows us to train grapheme-based, uni-directional attention-based
models which match the performance of a traditional, state-of-the-art,
discriminative sequence-trained system on a mobile voice-search task.
