  It is difficult for an E2E ASR system to recognize words such as entities
appearing infrequently in the training data. A widely used method to mitigate
this issue is feeding contextual information into the acoustic model. Previous
works have proven that a compact and accurate contextual list can boost the
performance significantly. In this paper, we propose an efficient approach to
obtain a high quality contextual list for a unified streaming/non-streaming
based E2E model. Specifically, we make use of the phone-level streaming output
to first filter the predefined contextual word list then fuse it into
non-casual encoder and decoder to generate the final recognition results. Our
approach improve the accuracy of the contextual ASR system and speed up the
inference process. Experiments on two datasets demonstrates over 20% CER
reduction comparing to the baseline system. Meanwhile, the RTF of our system
can be stabilized within 0.15 when the size of the contextual word list grows
over 6,000.
