  SqueezeFormer has recently shown impressive performance in automatic speech
recognition (ASR). However, its inference speed suffers from the quadratic
complexity of softmax-attention (SA). In addition, limited by the large
convolution kernel size, the local modeling ability of SqueezeFormer is
insufficient. In this paper, we propose a novel method HybridFormer to improve
SqueezeFormer in a fast and efficient way. Specifically, we first incorporate
linear attention (LA) and propose a hybrid LASA paradigm to increase the
model's inference speed. Second, a hybrid neural architecture search (NAS)
guided structural re-parameterization (SRep) mechanism, termed NSR, is proposed
to enhance the ability of the model to extract local interactions. Extensive
experiments conducted on the LibriSpeech dataset demonstrate that our proposed
HybridFormer can achieve a 9.1% relative word error rate (WER) reduction over
SqueezeFormer on the test-other dataset. Furthermore, when input speech is 30s,
the HybridFormer can improve the model's inference speed up to 18%. Our source
code is available online.
