  Thermal infrared (TIR) image has proven effectiveness in providing
temperature cues to the RGB features for multispectral pedestrian detection.
Most existing methods directly inject the TIR modality into the RGB-based
framework or simply ensemble the results of two modalities. This, however,
could lead to inferior detection performance, as the RGB and TIR features
generally have modality-specific noise, which might worsen the features along
with the propagation of the network. Therefore, this work proposes an effective
and efficient cross-modality fusion module called Bi-directional Adaptive
Attention Gate (BAA-Gate). Based on the attention mechanism, the BAA-Gate is
devised to distill the informative features and recalibrate the representations
asymptotically. Concretely, a bi-direction multi-stage fusion strategy is
adopted to progressively optimize features of two modalities and retain their
specificity during the propagation. Moreover, an adaptive interaction of
BAA-Gate is introduced by the illumination-based weighting strategy to
adaptively adjust the recalibrating and aggregating strength in the BAA-Gate
and enhance the robustness towards illumination changes. Considerable
experiments on the challenging KAIST dataset demonstrate the superior
performance of our method with satisfactory speed.
