  Generalized correlation analysis (GCA) is concerned with uncovering linear
relationships across multiple datasets. It generalizes canonical correlation
analysis that is designed for two datasets. We study sparse GCA when there are
potentially multiple generalized correlation tuples in data and the loading
matrix has a small number of nonzero rows. It includes sparse CCA and sparse
PCA of correlation matrices as special cases. We first formulate sparse GCA as
generalized eigenvalue problems at both population and sample levels via a
careful choice of normalization constraints. Based on a Lagrangian form of the
sample optimization problem, we propose a thresholded gradient descent
algorithm for estimating GCA loading vectors and matrices in high dimensions.
We derive tight estimation error bounds for estimators generated by the
algorithm with proper initialization. We also demonstrate the prowess of the
algorithm on a number of synthetic datasets.
