  This paper addresses the problem of cross-modal object tracking from RGB
videos and event data. Rather than constructing a complex cross-modal fusion
network, we explore the great potential of a pre-trained vision Transformer
(ViT). Particularly, we delicately investigate plug-and-play training
augmentations that encourage the ViT to bridge the vast distribution gap
between the two modalities, enabling comprehensive cross-modal information
interaction and thus enhancing its ability. Specifically, we propose a mask
modeling strategy that randomly masks a specific modality of some tokens to
enforce the interaction between tokens from different modalities interacting
proactively. To mitigate network oscillations resulting from the masking
strategy and further amplify its positive effect, we then theoretically propose
an orthogonal high-rank loss to regularize the attention matrix. Extensive
experiments demonstrate that our plug-and-play training augmentation techniques
can significantly boost state-of-the-art one-stream and twostream trackers to a
large extent in terms of both tracking precision and success rate. Our new
perspective and findings will potentially bring insights to the field of
leveraging powerful pre-trained ViTs to model cross-modal data. The code will
be publicly available.
