  Survival analysis, as a challenging task, requires integrating Whole Slide
Images (WSIs) and genomic data for comprehensive decision-making. There are two
main challenges in this task: significant heterogeneity and complex inter- and
intra-modal interactions between the two modalities. Previous approaches
utilize co-attention methods, which fuse features from both modalities only
once after separate encoding. However, these approaches are insufficient for
modeling the complex task due to the heterogeneous nature between the
modalities. To address these issues, we propose a Biased Progressive Encoding
(BPE) paradigm, performing encoding and fusion simultaneously. This paradigm
uses one modality as a reference when encoding the other. It enables deep
fusion of the modalities through multiple alternating iterations, progressively
reducing the cross-modal disparities and facilitating complementary
interactions. Besides modality heterogeneity, survival analysis involves
various biomarkers from WSIs, genomics, and their combinations. The critical
biomarkers may exist in different modalities under individual variations,
necessitating flexible adaptation of the models to specific scenarios.
Therefore, we further propose a Mixture of Multimodal Experts (MoME) layer to
dynamically selects tailored experts in each stage of the BPE paradigm. Experts
incorporate reference information from another modality to varying degrees,
enabling a balanced or biased focus on different modalities during the encoding
process. Extensive experimental results demonstrate the superior performance of
our method on various datasets, including TCGA-BLCA, TCGA-UCEC and TCGA-LUAD.
Codes are available at https://github.com/BearCleverProud/MoME.
