  Personalization of automatic speech recognition (ASR) models is a widely
studied topic because of its many practical applications. Most recently,
attention-based contextual biasing techniques are used to improve the
recognition of rare words and domain specific entities. However, due to
performance constraints, the biasing is often limited to a few thousand
entities, restricting real-world usability. To address this, we first propose a
"Retrieve and Copy" mechanism to improve latency while retaining the accuracy
even when scaled to a large catalog. We also propose a training strategy to
overcome the degradation in recall at such scale due to an increased number of
confusing entities. Overall, our approach achieves up to 6% more Word Error
Rate reduction (WERR) and 3.6% absolute improvement in F1 when compared to a
strong baseline. Our method also allows for large catalog sizes of up to 20K
without significantly affecting WER and F1-scores, while achieving at least 20%
inference speedup per acoustic frame.
