  Toward high-performance multilingual automatic speech recognition (ASR),
various types of linguistic information and model design have demonstrated
their effectiveness independently. They include language identity (LID),
phoneme information, language-specific processing modules, and cross-lingual
self-supervised speech representation. It is expected that leveraging their
benefits synergistically in a unified solution would further improve the
overall system performance. This paper presents a novel design of a
hierarchical information path, named LUPET, which sequentially encodes, from
the shallow layers to deep layers, multiple aspects of linguistic and acoustic
information at diverse granularity scales. The path starts from LID prediction,
followed by acoustic unit discovery, phoneme sharing, and finally token
recognition routed by a mixture-of-expert. ASR experiments are carried out on
10 languages in the Common Voice corpus. The results demonstrate the superior
performance of LUPET as compared to the baseline systems. Most importantly,
LUPET effectively mitigates the issue of performance compromise of
high-resource languages with low-resource ones in the multilingual setting.
