  End-to-end model, especially Recurrent Neural Network Transducer (RNN-T), has
achieved great success in speech recognition. However, transducer requires a
great memory footprint and computing time when processing a long decoding
sequence. To solve this problem, we propose a model named time-sparse
transducer, which introduces a time-sparse mechanism into transducer. In this
mechanism, we obtain the intermediate representations by reducing the time
resolution of the hidden states. Then the weighted average algorithm is used to
combine these representations into sparse hidden states followed by the
decoder. All the experiments are conducted on a Mandarin dataset AISHELL-1.
Compared with RNN-T, the character error rate of the time-sparse transducer is
close to RNN-T and the real-time factor is 50.00% of the original. By adjusting
the time resolution, the time-sparse transducer can also reduce the real-time
factor to 16.54% of the original at the expense of a 4.94% loss of precision.
