  Effectively leveraging multimodal data such as various images, laboratory
tests and clinical information is gaining traction in a variety of AI-based
medical diagnosis and prognosis tasks. Most existing multi-modal techniques
only focus on enhancing their performance by leveraging the differences or
shared features from various modalities and fusing feature across different
modalities. These approaches are generally not optimal for clinical settings,
which pose the additional challenges of limited training data, as well as being
rife with redundant data or noisy modality channels, leading to subpar
performance. To address this gap, we study the robustness of existing methods
to data redundancy and noise and propose a generalized dynamic multimodal
information bottleneck framework for attaining a robust fused feature
representation. Specifically, our information bottleneck module serves to
filter out the task-irrelevant information and noises in the fused feature, and
we further introduce a sufficiency loss to prevent dropping of task-relevant
information, thus explicitly preserving the sufficiency of prediction
information in the distilled feature. We validate our model on an in-house and
a public COVID19 dataset for mortality prediction as well as two public
biomedical datasets for diagnostic tasks. Extensive experiments show that our
method surpasses the state-of-the-art and is significantly more robust, being
the only method to remain performance when large-scale noisy channels exist.
Our code is publicly available at https://github.com/ayanglab/DMIB.
