  We present Deep Tensor Canonical Correlation Analysis (DTCCA), a method to
learn complex nonlinear transformations of multiple views (more than two) of
data such that the resulting representations are linearly correlated in high
order. The high-order correlation of given multiple views is modeled by
covariance tensor, which is different from most CCA formulations relying solely
on the pairwise correlations. Parameters of transformations of each view are
jointly learned by maximizing the high-order canonical correlation. To solve
the resulting problem, we reformulate it as the best sum of rank-1
approximation, which can be efficiently solved by existing tensor decomposition
method. DTCCA is a nonlinear extension of tensor CCA (TCCA) via deep networks.
The transformations of DTCCA are parametric functions, which are very different
from implicit mapping in the form of kernel function. Comparing with kernel
TCCA, DTCCA not only can deal with arbitrary dimensions of the input data, but
also does not need to maintain the training data for computing representations
of any given data point. Hence, DTCCA as a unified model can efficiently
overcome the scalable issue of TCCA for either high-dimensional multi-view data
or a large amount of views, and it also naturally extends TCCA for learning
nonlinear representation. Extensive experiments on three multi-view data sets
demonstrate the effectiveness of the proposed method.
