  We propose an adaptation method for factorized neural transducers (FNT) with
external language models. We demonstrate that both neural and n-gram external
LMs add significantly more value when linearly interpolated with predictor
output compared to shallow fusion, thus confirming that FNT forces the
predictor to act like regular language models. Further, we propose a method to
integrate class-based n-gram language models into FNT framework resulting in
accuracy gains similar to a hybrid setup. We show average gains of 18% WERR
with lexical adaptation across various scenarios and additive gains of up to
60% WERR in one entity-rich scenario through a combination of class-based
n-gram and neural LMs.
