  Predicting in-hospital mortality for intensive care unit (ICU) patients is
key to final clinical outcomes. AI has shown advantaged accuracy but suffers
from the lack of explainability. To address this issue, this paper proposes an
eXplainable Multimodal Mortality Predictor (X-MMP) approaching an efficient,
explainable AI solution for predicting in-hospital mortality via multimodal ICU
data. We employ multimodal learning in our framework, which can receive
heterogeneous inputs from clinical data and make decisions. Furthermore, we
introduce an explainable method, namely Layer-Wise Propagation to Transformer,
as a proper extension of the LRP method to Transformers, producing explanations
over multimodal inputs and revealing the salient features attributed to
prediction. Moreover, the contribution of each modality to clinical outcomes
can be visualized, assisting clinicians in understanding the reasoning behind
decision-making. We construct a multimodal dataset based on MIMIC-III and
MIMIC-III Waveform Database Matched Subset. Comprehensive experiments on
benchmark datasets demonstrate that our proposed framework can achieve
reasonable interpretation with competitive prediction accuracy. In particular,
our framework can be easily transferred to other clinical tasks, which
facilitates the discovery of crucial factors in healthcare research.
