  Existing work in intelligent communications has recently made preliminary
attempts to utilize multi-source sensing information (MSI) to improve the
system performance. However, the research on MSI aided intelligent
communications has not yet explored how to integrate and fuse the multimodal
sensory data, which motivates us to develop a systematic framework for wireless
communications based on deep multimodal learning (DML). In this paper, we first
present complete descriptions and heuristic understandings on the framework of
DML based wireless communications, where core design choices are analyzed in
the view of communications. Then, we develop several DML based architectures
for channel prediction in massive multiple-input multiple-output (MIMO) systems
that leverage various modality combinations and fusion levels. The case study
of massive MIMO channel prediction offers an important example that can be
followed in developing other DML based communication technologies. Simulations
results demonstrate that the proposed DML framework can effectively exploit the
constructive and complementary information of multimodal sensory data in
various wireless communication scenarios.
