  This paper proposes a modification to RNN-Transducer (RNN-T) models for
automatic speech recognition (ASR). In standard RNN-T, the emission of a blank
symbol consumes exactly one input frame; in our proposed method, we introduce
additional blank symbols, which consume two or more input frames when emitted.
We refer to the added symbols as big blanks, and the method multi-blank RNN-T.
For training multi-blank RNN-Ts, we propose a novel logit under-normalization
method in order to prioritize emissions of big blanks. With experiments on
multiple languages and datasets, we show that multi-blank RNN-T methods could
bring relative speedups of over +90%/+139% to model inference for English
Librispeech and German Multilingual Librispeech datasets, respectively. The
multi-blank RNN-T method also improves ASR accuracy consistently. We will
release our implementation of the method in the NeMo
(https://github.com/NVIDIA/NeMo) toolkit.
