  In end-to-end automatic speech recognition system, one of the difficulties
for language expansion is the limited paired speech and text training data. In
this paper, we propose a novel method to generate augmented samples with
unpaired speech feature segments and text data for model pre-training, which
has the advantage of low cost without using additional speech data. When mixing
20,000 hours augmented speech data generated by our method with 12,500 hours
original transcribed speech data for Italian Transformer transducer model
pre-training, we achieve 8.7% relative word error rate reduction. The
pre-trained model achieves similar performance as the model pre-trained with
multilingual transcribed 75,000 hours raw speech data. When merging the
augmented speech data with the multilingual data to pre-train a new model, we
achieve even more relative word error rate reduction of 12.2% over the
baseline, which further verifies the effectiveness of our method for speech
data augmentation.
