  In this paper, we address the problem of hidden common variables discovery
from multimodal data sets of nonlinear high-dimensional observations. We
present a metric based on local applications of canonical correlation analysis
(CCA) and incorporate it in a kernel-based manifold learning technique.We show
that this metric discovers the hidden common variables underlying the
multimodal observations by estimating the Euclidean distance between them. Our
approach can be viewed both as an extension of CCA to a nonlinear setting as
well as an extension of manifold learning to multiple data sets. Experimental
results show that our method indeed discovers the common variables underlying
high-dimensional nonlinear observations without assuming prior rigid model
assumptions.
