  Empowered by transformer-based models, visual tracking has advanced
significantly. However, the slow speed of current trackers limits their
applicability on devices with constrained computational resources. To address
this challenge, we introduce ABTrack, an adaptive computation framework that
adaptively bypassing transformer blocks for efficient visual tracking. The
rationale behind ABTrack is rooted in the observation that semantic features or
relations do not uniformly impact the tracking task across all abstraction
levels. Instead, this impact varies based on the characteristics of the target
and the scene it occupies. Consequently, disregarding insignificant semantic
features or relations at certain abstraction levels may not significantly
affect the tracking accuracy. We propose a Bypass Decision Module (BDM) to
determine if a transformer block should be bypassed, which adaptively
simplifies the architecture of ViTs and thus speeds up the inference process.
To counteract the time cost incurred by the BDMs and further enhance the
efficiency of ViTs, we introduce a novel ViT pruning method to reduce the
dimension of the latent representation of tokens in each transformer block.
Extensive experiments on multiple tracking benchmarks validate the
effectiveness and generality of the proposed method and show that it achieves
state-of-the-art performance. Code is released at:
https://github.com/xyyang317/ABTrack.
