  Recently, there has been increasing progress in end-to-end automatic speech
recognition (ASR) architecture, which transcribes speech to text without any
pre-trained alignments. One popular end-to-end approach is the hybrid
Connectionist Temporal Classification (CTC) and attention (CTC/attention) based
ASR architecture. However, how to deploy hybrid CTC/attention systems for
online speech recognition is still a non-trivial problem. This article
describes our proposed online hybrid CTC/attention end-to-end ASR architecture,
which replaces all the offline components of conventional CTC/attention ASR
architecture with their corresponding streaming components. Firstly, we propose
stable monotonic chunk-wise attention (sMoChA) to stream the conventional
global attention, and further propose monotonic truncated attention (MTA) to
simplify sMoChA and solve the training-and-decoding mismatch problem of sMoChA.
Secondly, we propose truncated CTC (T-CTC) prefix score to stream CTC prefix
score calculation. Thirdly, we design dynamic waiting joint decoding (DWJD)
algorithm to dynamically collect the predictions of CTC and attention in an
online manner. Finally, we use latency-controlled bidirectional long short-term
memory (LC-BLSTM) to stream the widely-used offline bidirectional encoder
network. Experiments with LibriSpeech English and HKUST Mandarin tasks
demonstrate that, compared with the offline CTC/attention model, our proposed
online CTC/attention model improves the real time factor in human-computer
interaction services and maintains its performance with moderate degradation.
To the best of our knowledge, this is the first work to provide the full-stack
online solution for CTC/attention end-to-end ASR architecture.
