  Infrared-visible object detection aims to achieve robust even full-day object
detection by fusing the complementary information of infrared and visible
images. However, highly dynamically variable complementary characteristics and
commonly existing modality misalignment make the fusion of complementary
information difficult. In this paper, we propose a Dynamic Adaptive
Multispectral Detection Transformer (DAMSDet) to simultaneously address these
two challenges. Specifically, we propose a Modality Competitive Query Selection
strategy to provide useful prior information. This strategy can dynamically
select basic salient modality feature representation for each object. To
effectively mine the complementary information and adapt to misalignment
situations, we propose a Multispectral Deformable Cross-attention module to
adaptively sample and aggregate multi-semantic level features of infrared and
visible images for each object. In addition, we further adopt the cascade
structure of DETR to better mine complementary information. Experiments on four
public datasets of different scenes demonstrate significant improvements
compared to other state-of-the-art methods. The code will be released at
https://github.com/gjj45/DAMSDet.
