  Multi-modal fusion is crucial in medical data research, enabling a
comprehensive understanding of diseases and improving diagnostic performance by
combining diverse modalities. However, multi-modal fusion faces challenges,
including capturing interactions between modalities, addressing missing
modalities, handling erroneous modal information, and ensuring
interpretability. Many existing researchers tend to design different solutions
for these problems, often overlooking the commonalities among them. This paper
proposes a novel multi-modal fusion framework that achieves adaptive adjustment
over the weights of each modality by introducing the Modal-Domain Attention
(MDA). It aims to facilitate the fusion of multi-modal information while
allowing for the inclusion of missing modalities or intrinsic noise, thereby
enhancing the representation of multi-modal data. We provide visualizations of
accuracy changes and MDA weights by observing the process of modal fusion,
offering a comprehensive analysis of its interpretability. Extensive
experiments on various gastrointestinal disease benchmarks, the proposed MDA
maintains high accuracy even in the presence of missing modalities and
intrinsic noise. One thing worth mentioning is that the visualization of MDA is
highly consistent with the conclusions of existing clinical studies on the
dependence of different diseases on various modalities. Code and dataset will
be made available.
