  Deploying end-to-end speech recognition models with limited computing
resources remains challenging, despite their impressive performance. Given the
gradual increase in model size and the wide range of model applications,
selectively executing model components for different inputs to improve the
inference efficiency is of great interest. In this paper, we propose a dynamic
layer-skipping method that leverages the CTC blank output from intermediate
layers to trigger the skipping of the last few encoder layers for frames with
high blank probabilities. Furthermore, we factorize the CTC output distribution
and perform knowledge distillation on intermediate layers to reduce computation
and improve recognition accuracy. Experimental results show that by utilizing
the CTC blank, the encoder layer depth can be adjusted dynamically, resulting
in 29% acceleration of the CTC model inference with minor performance
degradation.
