  The Transformer self-attention network has shown promising performance as an
alternative to recurrent neural networks in end-to-end (E2E) automatic speech
recognition (ASR) systems. However, Transformer has a drawback in that the
entire input sequence is required to compute both self-attention and
source--target attention. In this paper, we propose a novel blockwise
synchronous beam search algorithm based on blockwise processing of encoder to
perform streaming E2E Transformer ASR. In the beam search, encoded feature
blocks are synchronously aligned using a block boundary detection technique,
where a reliability score of each predicted hypothesis is evaluated based on
the end-of-sequence and repeated tokens in the hypothesis. Evaluations of the
HKUST and AISHELL-1 Mandarin, LibriSpeech English, and CSJ Japanese tasks show
that the proposed streaming Transformer algorithm outperforms conventional
online approaches, including monotonic chunkwise attention (MoChA), especially
when using the knowledge distillation technique. An ablation study indicates
that our streaming approach contributes to reducing the response time, and the
repetition criterion contributes significantly in certain tasks. Our streaming
ASR models achieve comparable or superior performance to batch models and other
streaming-based Transformer methods in all tasks considered.
