  Visual object tracking, which is primarily based on visible light image
sequences, encounters numerous challenges in complicated scenarios, such as low
light conditions, high dynamic ranges, and background clutter. To address these
challenges, incorporating the advantages of multiple visual modalities is a
promising solution for achieving reliable object tracking. However, the
existing approaches usually integrate multimodal inputs through adaptive local
feature interactions, which cannot leverage the full potential of visual cues,
thus resulting in insufficient feature modeling. In this study, we propose a
novel multimodal hybrid tracker (MMHT) that utilizes frame-event-based data for
reliable single object tracking. The MMHT model employs a hybrid backbone
consisting of an artificial neural network (ANN) and a spiking neural network
(SNN) to extract dominant features from different visual modalities and then
uses a unified encoder to align the features across different domains.
Moreover, we propose an enhanced transformer-based module to fuse multimodal
features using attention mechanisms. With these methods, the MMHT model can
effectively construct a multiscale and multidimensional visual feature space
and achieve discriminative feature modeling. Extensive experiments demonstrate
that the MMHT model exhibits competitive performance in comparison with that of
other state-of-the-art methods. Overall, our results highlight the
effectiveness of the MMHT model in terms of addressing the challenges faced in
visual object tracking tasks.
