  Healthcare data are inherently multimodal, including electronic health
records (EHR), medical images, and multi-omics data. Combining these multimodal
data sources contributes to a better understanding of human health and provides
optimal personalized healthcare. Advances in artificial intelligence (AI)
technologies, particularly machine learning (ML), enable the fusion of these
different data modalities to provide multimodal insights. To this end, in this
scoping review, we focus on synthesizing and analyzing the literature that uses
AI techniques to fuse multimodal medical data for different clinical
applications. More specifically, we focus on studies that only fused EHR with
medical imaging data to develop various AI methods for clinical applications.
We present a comprehensive analysis of the various fusion strategies, the
diseases and clinical outcomes for which multimodal fusion was used, the ML
algorithms used to perform multimodal fusion for each clinical application, and
the available multimodal medical datasets. We followed the PRISMA-ScR
guidelines. We searched Embase, PubMed, Scopus, and Google Scholar to retrieve
relevant studies. We extracted data from 34 studies that fulfilled the
inclusion criteria. In our analysis, a typical workflow was observed: feeding
raw data, fusing different data modalities by applying conventional machine
learning (ML) or deep learning (DL) algorithms, and finally, evaluating the
multimodal fusion through clinical outcome predictions. Specifically, early
fusion was the most used technique in most applications for multimodal learning
(22 out of 34 studies). We found that multimodality fusion models outperformed
traditional single-modality models for the same task. Disease diagnosis and
prediction were the most common clinical outcomes (reported in 20 and 10
studies, respectively) from a clinical outcome perspective.
