  End-to-end models have been showing superiority in Automatic Speech
Recognition (ASR). At the same time, the capacity of streaming recognition has
become a growing requirement for end-to-end models. Following these trends, an
encoder-decoder recurrent neural network called Recurrent Neural Aligner (RNA)
has been freshly proposed and shown its competitiveness on two English ASR
tasks. However, it is not clear if RNA can be further improved and applied to
other spoken language. In this work, we explore the applicability of RNA in
Mandarin Chinese and present four effective extensions: In the encoder, we
redesign the temporal down-sampling and introduce a powerful convolutional
structure. In the decoder, we utilize a regularizer to smooth the output
distribution and conduct joint training with a language model. On two Mandarin
Chinese conversational telephone speech recognition (MTS) datasets, our
Extended-RNA obtains promising performance. Particularly, it achieves 27.7%
character error rate (CER), which is superior to current state-of-the-art
result on the popular HKUST task.
