  Representation learning is typically applied to only one mode of a data
matrix, either its rows or columns. Yet in many applications, there is an
underlying geometry to both the rows and the columns. We propose utilizing this
coupled structure to perform co-manifold learning: uncovering the underlying
geometry of both the rows and the columns of a given matrix, where we focus on
a missing data setting. Our unsupervised approach consists of three components.
We first solve a family of optimization problems to estimate a complete matrix
at multiple scales of smoothness. We then use this collection of smooth matrix
estimates to compute pairwise distances on the rows and columns based on a new
multi-scale metric that implicitly introduces a coupling between the rows and
the columns. Finally, we construct row and column representations from these
multi-scale metrics. We demonstrate that our approach outperforms competing
methods in both data visualization and clustering.
