  Current mainstream vision-language (VL) tracking framework consists of three
parts, \ie a visual feature extractor, a language feature extractor, and a
fusion model. To pursue better performance, a natural modus operandi for VL
tracking is employing customized and heavier unimodal encoders, and multi-modal
fusion models. Albeit effective, existing VL trackers separate feature
extraction and feature integration, resulting in extracted features that lack
semantic guidance and have limited target-aware capability in complex
scenarios, \eg similar distractors and extreme illumination. In this work,
inspired by the recent success of exploring foundation models with unified
architecture for both natural language and computer vision tasks, we propose an
All-in-One framework, which learns joint feature extraction and interaction by
adopting a unified transformer backbone. Specifically, we mix raw vision and
language signals to generate language-injected vision tokens, which we then
concatenate before feeding into the unified backbone architecture. This
approach achieves feature integration in a unified backbone, removing the need
for carefully-designed fusion modules and resulting in a more effective and
efficient VL tracking framework. To further improve the learning efficiency, we
introduce a multi-modal alignment module based on cross-modal and intra-modal
contrastive objectives, providing more reasonable representations for the
unified All-in-One transformer backbone. Extensive experiments on five
benchmarks, \ie OTB99-L, TNL2K, LaSOT, LaSOT$_{\rm Ext}$ and WebUAV-3M,
demonstrate the superiority of the proposed tracker against existing
state-of-the-arts on VL tracking. Codes will be made publicly available.
