  A streaming style inference of encoder-decoder automatic speech recognition
(ASR) system is important for reducing latency, which is essential for
interactive use cases. To this end, we propose a novel blockwise synchronous
decoding algorithm with a hybrid approach that combines endpoint prediction and
endpoint post-determination. In the endpoint prediction, we compute the
expectation of the number of tokens that are yet to be emitted in the encoder
features of the current blocks using the CTC posterior. Based on the
expectation value, the decoder predicts the endpoint to realize continuous
block synchronization, as a running stitch. Meanwhile, endpoint
post-determination probabilistically detects backward jump of the source-target
attention, which is caused by the misprediction of endpoints. Then it resumes
decoding by discarding those hypotheses, as back stitch. We combine these
methods into a hybrid approach, namely run-and-back stitch search, which
reduces the computational cost and latency. Evaluations of various ASR tasks
show the efficiency of our proposed decoding algorithm, which achieves a
latency reduction, for instance in the Librispeech test set from 1487 ms to 821
ms at the 90th percentile, while maintaining a high recognition accuracy.
