  In this paper, we propose a spectral method for deriving functions that are
jointly smooth on multiple observed manifolds. This allows us to register
measurements of the same phenomenon by heterogeneous sensors, and to reject
sensor-specific noise. Our method is unsupervised and primarily consists of two
steps. First, using kernels, we obtain a subspace spanning smooth functions on
each separate manifold. Then, we apply a spectral method to the obtained
subspaces and discover functions that are jointly smooth on all manifolds. We
show analytically that our method is guaranteed to provide a set of orthogonal
functions that are as jointly smooth as possible, ordered by increasing
Dirichlet energy from the smoothest to the least smooth. In addition, we show
that the extracted functions can be efficiently extended to unseen data using
the Nystr\"{o}m method. We demonstrate the proposed method on both simulated
and real measured data and compare the results to nonlinear variants of the
seminal Canonical Correlation Analysis (CCA). Particularly, we show superior
results for sleep stage identification. In addition, we show how the proposed
method can be leveraged for finding minimal realizations of parameter spaces of
nonlinear dynamical systems.
