  Survival prediction is a complicated ordinal regression task that aims to
predict the ranking risk of death, which generally benefits from the
integration of histology and genomic data. Despite the progress in joint
learning from pathology and genomics, existing methods still suffer from
challenging issues: 1) Due to the large size of pathological images, it is
difficult to effectively represent the gigapixel whole slide images (WSIs). 2)
Interactions within tumor microenvironment (TME) in histology are essential for
survival analysis. Although current approaches attempt to model these
interactions via co-attention between histology and genomic data, they focus on
only dense local similarity across modalities, which fails to capture global
consistency between potential structures, i.e. TME-related interactions of
histology and co-expression of genomic data. To address these challenges, we
propose a Multimodal Optimal Transport-based Co-Attention Transformer framework
with global structure consistency, in which optimal transport (OT) is applied
to match patches of a WSI and genes embeddings for selecting informative
patches to represent the gigapixel WSI. More importantly, OT-based co-attention
provides a global awareness to effectively capture structural interactions
within TME for survival prediction. To overcome high computational complexity
of OT, we propose a robust and efficient implementation over micro-batch of WSI
patches by approximating the original OT with unbalanced mini-batch OT.
Extensive experiments show the superiority of our method on five benchmark
datasets compared to the state-of-the-art methods. The code is released.
