  This paper presents a Tri-branch Neural Fusion (TNF) approach designed for
classifying multimodal medical images and tabular data. It also introduces two
solutions to address the challenge of label inconsistency in multimodal
classification. Traditional methods in multi-modality medical data
classification often rely on single-label approaches, typically merging
features from two distinct input modalities. This becomes problematic when
features are mutually exclusive or labels differ across modalities, leading to
reduced accuracy. To overcome this, our TNF approach implements a tri-branch
framework that manages three separate outputs: one for image modality, another
for tabular modality, and a third hybrid output that fuses both image and
tabular data. The final decision is made through an ensemble method that
integrates likelihoods from all three branches. We validate the effectiveness
of TNF through extensive experiments, which illustrate its superiority over
traditional fusion and ensemble methods in various convolutional neural
networks and transformer-based architectures across multiple datasets.
