  Despite the success of deep learning in speech recognition, multi-dialect
speech recognition remains a difficult problem. Although dialect-specific
acoustic models are known to perform well in general, they are not easy to
maintain when dialect-specific data is scarce and the number of dialects for
each language is large. Therefore, a single unified acoustic model (AM) that
generalizes well for many dialects has been in demand. In this paper, we
propose a novel acoustic modeling technique for accurate multi-dialect speech
recognition with a single AM. Our proposed AM is dynamically adapted based on
both dialect information and its internal representation, which results in a
highly adaptive AM for handling multiple dialects simultaneously. We also
propose a simple but effective training method to deal with unseen dialects.
The experimental results on large scale speech datasets show that the proposed
AM outperforms all the previous ones, reducing word error rates (WERs) by 8.11%
relative compared to a single all-dialects AM and by 7.31% relative compared to
dialect-specific AMs.
