  In this work, we investigate two popular end-to-end automatic speech
recognition (ASR) models, namely Connectionist Temporal Classification (CTC)
and RNN-Transducer (RNN-T), for offline recognition of voice search queries,
with up to 2B model parameters. The encoders of our models use the neural
architecture of Google's universal speech model (USM), with additional funnel
pooling layers to significantly reduce the frame rate and speed up training and
inference. We perform extensive studies on vocabulary size, time reduction
strategy, and its generalization performance on long-form test sets. Despite
the speculation that, as the model size increases, CTC can be as good as RNN-T
which builds label dependency into the prediction, we observe that a 900M RNN-T
clearly outperforms a 1.8B CTC and is more tolerant to severe time reduction,
although the WER gap can be largely removed by LM shallow fusion.
