  Multimodal data arise in various applications where information about the
same phenomenon is acquired from multiple sensors and across different imaging
modalities. Learning from multimodal data is of great interest in machine
learning and statistics research as this offers the possibility of capturing
complementary information among modalities. Multimodal modeling helps to
explain the interdependence between heterogeneous data sources, discovers new
insights that may not be available from a single modality, and improves
decision-making. Recently, coupled matrix-tensor factorization has been
introduced for multimodal data fusion to jointly estimate latent factors and
identify complex interdependence among the latent factors. However, most of the
prior work on coupled matrix-tensor factors focuses on unsupervised learning
and there is little work on supervised learning using the jointly estimated
latent factors. This paper considers the multimodal tensor data classification
problem. A Coupled Support Tensor Machine (C-STM) built upon the latent factors
jointly estimated from the Advanced Coupled Matrix Tensor Factorization (ACMTF)
is proposed. C-STM combines individual and shared latent factors with multiple
kernels and estimates a maximal-margin classifier for coupled matrix tensor
data. The classification risk of C-STM is shown to converge to the optimal
Bayes risk, making it a statistically consistent rule. C-STM is validated
through simulation studies as well as a simultaneous EEG-fMRI analysis. The
empirical evidence shows that C-STM can utilize information from multiple
sources and provide a better classification performance than traditional
single-mode classifiers.
