  Multispectral pedestrian detection has been shown to be effective in
improving performance within complex illumination scenarios. However, prevalent
double-stream networks in multispectral detection employ two separate feature
extraction branches for multi-modal data, leading to nearly double the
inference time compared to single-stream networks utilizing only one feature
extraction branch. This increased inference time has hindered the widespread
employment of multispectral pedestrian detection in embedded devices for
autonomous systems. To address this limitation, various knowledge distillation
methods have been proposed. However, traditional distillation methods focus
only on the fusion features and ignore the large amount of information in the
original multi-modal features, thereby restricting the student network's
performance. To tackle the challenge, we introduce the Adaptive Modal Fusion
Distillation (AMFD) framework, which can fully utilize the original modal
features of the teacher network. Specifically, a Modal Extraction Alignment
(MEA) module is utilized to derive learning weights for student networks,
integrating focal and global attention mechanisms. This methodology enables the
student network to acquire optimal fusion strategies independent from that of
teacher network without necessitating an additional feature fusion module.
Furthermore, we present the SMOD dataset, a well-aligned challenging
multispectral dataset for detection. Extensive experiments on the challenging
KAIST, LLVIP and SMOD datasets are conducted to validate the effectiveness of
AMFD. The results demonstrate that our method outperforms existing
state-of-the-art methods in both reducing log-average Miss Rate and improving
mean Average Precision. The code is available at
https://github.com/bigD233/AMFD.git.
