  Integrating cross-department multi-modal data (e.g., radiological,
pathological, genomic, and clinical data) is ubiquitous in brain cancer
diagnosis and survival prediction. To date, such an integration is typically
conducted by human physicians (and panels of experts), which can be subjective
and semi-quantitative. Recent advances in multi-modal deep learning, however,
have opened a door to leverage such a process to a more objective and
quantitative manner. Unfortunately, the prior arts of using four modalities on
brain cancer survival prediction are limited by a "complete modalities" setting
(i.e., with all modalities available). Thus, there are still open questions on
how to effectively predict brain cancer survival from the incomplete
radiological, pathological, genomic, and demographic data (e.g., one or more
modalities might not be collected for a patient). For instance, should we use
both complete and incomplete data, and more importantly, how to use those data?
To answer the preceding questions, we generalize the multi-modal learning on
cross-department multi-modal data to a missing data setting. Our contribution
is three-fold: 1) We introduce optimal multi-modal learning with missing data
(MMD) pipeline with optimized hardware consumption and computational
efficiency; 2) We extend multi-modal learning on radiological, pathological,
genomic, and demographic data into missing data scenarios; 3) a large-scale
public dataset (with 962 patients) is collected to systematically evaluate
glioma tumor survival prediction using four modalities. The proposed method
improved the C-index of survival prediction from 0.7624 to 0.8053.
