  In this paper we propose a novel data augmentation method for attention-based
end-to-end automatic speech recognition (E2E-ASR), utilizing a large amount of
text which is not paired with speech signals. Inspired by the back-translation
technique proposed in the field of machine translation, we build a neural
text-to-encoder model which predicts a sequence of hidden states extracted by a
pre-trained E2E-ASR encoder from a sequence of characters. By using hidden
states as a target instead of acoustic features, it is possible to achieve
faster attention learning and reduce computational cost, thanks to sub-sampling
in E2E-ASR encoder, also the use of the hidden states can avoid to model
speaker dependencies unlike acoustic features. After training, the
text-to-encoder model generates the hidden states from a large amount of
unpaired text, then E2E-ASR decoder is retrained using the generated hidden
states as additional training data. Experimental evaluation using LibriSpeech
dataset demonstrates that our proposed method achieves improvement of ASR
performance and reduces the number of unknown words without the need for paired
data.
