  Sequence-to-sequence attention-based models integrate an acoustic,
pronunciation and language model into a single neural network, which make them
very suitable for multilingual automatic speech recognition (ASR). In this
paper, we are concerned with multilingual speech recognition on low-resource
languages by a single Transformer, one of sequence-to-sequence attention-based
models. Sub-words are employed as the multilingual modeling unit without using
any pronunciation lexicon. First, we show that a single multilingual ASR
Transformer performs well on low-resource languages despite of some language
confusion. We then look at incorporating language information into the model by
inserting the language symbol at the beginning or at the end of the original
sub-words sequence under the condition of language information being known
during training. Experiments on CALLHOME datasets demonstrate that the
multilingual ASR Transformer with the language symbol at the end performs
better and can obtain relatively 10.5\% average word error rate (WER) reduction
compared to SHL-MLSTM with residual learning. We go on to show that, assuming
the language information being known during training and testing, about
relatively 12.4\% average WER reduction can be observed compared to SHL-MLSTM
with residual learning through giving the language symbol as the sentence start
token.
