  Multispectral pedestrian detection is a technology designed to detect and
locate pedestrians in Color and Thermal images, which has been widely used in
automatic driving, video surveillance, etc. So far most available multispectral
pedestrian detection algorithms only achieved limited success in pedestrian
detection because of the lacking take into account the confusion of pedestrian
information and background noise in Color and Thermal images. Here we propose a
multispectral pedestrian detection algorithm, which mainly consists of a
cascaded information enhancement module and a cross-modal attention feature
fusion module. On the one hand, the cascaded information enhancement module
adopts the channel and spatial attention mechanism to perform attention
weighting on the features fused by the cascaded feature fusion block. Moreover,
it multiplies the single-modal features with the attention weight element by
element to enhance the pedestrian features in the single-modal and thus
suppress the interference from the background. On the other hand, the
cross-modal attention feature fusion module mines the features of both Color
and Thermal modalities to complement each other, then the global features are
constructed by adding the cross-modal complemented features element by element,
which are attentionally weighted to achieve the effective fusion of the two
modal features. Finally, the fused features are input into the detection head
to detect and locate pedestrians. Extensive experiments have been performed on
two improved versions of annotations (sanitized annotations and paired
annotations) of the public dataset KAIST. The experimental results show that
our method demonstrates a lower pedestrian miss rate and more accurate
pedestrian detection boxes compared to the comparison method. Additionally, the
ablation experiment also proved the effectiveness of each module designed in
this paper.
