  Cross-modality images that integrate visible-infrared spectra cues can
provide richer complementary information for object detection. Despite this,
existing visible-infrared object detection methods severely degrade in severe
weather conditions. This failure stems from the pronounced sensitivity of
visible images to environmental perturbations, such as rain, haze, and snow,
which frequently cause false negatives and false positives in detection. To
address this issue, we introduce a novel and challenging task, termed
visible-infrared object detection under adverse weather conditions. To foster
this task, we have constructed a new Severe Weather Visible-Infrared Dataset
(SWVID) with diverse severe weather scenes. Furthermore, we introduce the
Cross-modality Fusion Mamba with Weather-removal (CFMW) to augment detection
accuracy in adverse weather conditions. Thanks to the proposed Weather Removal
Diffusion Model (WRDM) and Cross-modality Fusion Mamba (CFM) modules, CFMW is
able to mine more essential information of pedestrian features in
cross-modality fusion, thus could transfer to other rarer scenarios with high
efficiency and has adequate availability on those platforms with low computing
power. To the best of our knowledge, this is the first study that targeted
improvement and integrated both Diffusion and Mamba modules in cross-modality
object detection, successfully expanding the practical application of this type
of model with its higher accuracy and more advanced architecture. Extensive
experiments on both well-recognized and self-created datasets conclusively
demonstrate that our CFMW achieves state-of-the-art detection performance,
surpassing existing benchmarks. The dataset and source code will be made
publicly available at https://github.com/lhy-zjut/CFMW.
