  In this paper, we present NECHO v2, a novel framework designed to enhance the
predictive accuracy of multimodal sequential patient diagnoses under uncertain
missing visit sequences, a common challenge in clinical settings. Firstly, we
modify NECHO to handle uncertain modality representation dominance under the
imperfect data. Next, we develop a systematic knowledge distillation by
employing the modified NECHO as both teacher and student. It encompasses a
modality-wise contrastive and hierarchical distillation, transformer
representation random distillation, along with other distillations to align
representations tightly and effectively. We also utilise random erasing on
individual data points within sequences during both training and distillation
of teacher to lightly simulate scenario with missing visit information to
foster effective knowledge transfer. As a result, NECHO v2 verifies itself by
showing superiority in multimodal sequential diagnosis prediction on both
balanced and imbalanced incomplete settings on multimodal healthcare data.
