  Object detection in Remote Sensing Images (RSI) is a critical task for
numerous applications in Earth Observation (EO). Differing from object
detection in natural images, object detection in remote sensing images faces
challenges of scarcity of annotated data and the presence of small objects
represented by only a few pixels. Multi-modal fusion has been determined to
enhance the accuracy by fusing data from multiple modalities such as RGB,
infrared (IR), lidar, and synthetic aperture radar (SAR). To this end, the
fusion of representations at the mid or late stage, produced by parallel
subnetworks, is dominant, with the disadvantages of increasing computational
complexity in the order of the number of modalities and the creation of
additional engineering obstacles. Using the cross-attention mechanism, we
propose a novel multi-modal fusion strategy for mapping relationships between
different channels at the early stage, enabling the construction of a coherent
input by aligning the different modalities. By addressing fusion in the early
stage, as opposed to mid or late-stage methods, our method achieves competitive
and even superior performance compared to existing techniques. Additionally, we
enhance the SWIN transformer by integrating convolution layers into the
feed-forward of non-shifting blocks. This augmentation strengthens the model's
capacity to merge separated windows through local attention, thereby improving
small object detection. Extensive experiments prove the effectiveness of the
proposed multimodal fusion module and the architecture, demonstrating their
applicability to object detection in multimodal aerial imagery.
