  Reducing prediction delay for streaming end-to-end ASR models with minimal
performance regression is a challenging problem. Constrained alignment is a
well-known existing approach that penalizes predicted word boundaries using
external low-latency acoustic models. On the contrary, recently proposed
FastEmit is a sequence-level delay regularization scheme encouraging vocabulary
tokens over blanks without any reference alignments. Although all these schemes
are successful in reducing delay, ASR word error rate (WER) often severely
degrades after applying these delay constraining schemes. In this paper, we
propose a novel delay constraining method, named self alignment. Self alignment
does not require external alignment models. Instead, it utilizes Viterbi
forced-alignments from the trained model to find the lower latency alignment
direction. From LibriSpeech evaluation, self alignment outperformed existing
schemes: 25% and 56% less delay compared to FastEmit and constrained alignment
at the similar word error rate. For Voice Search evaluation,12% and 25% delay
reductions were achieved compared to FastEmit and constrained alignment with
more than 2% WER improvements.
