  Probabilistic principal component analysis (PPCA) seeks a low dimensional
representation of a data set in the presence of independent spherical Gaussian
noise, Sigma = (sigma^2)*I. The maximum likelihood solution for the model is an
eigenvalue problem on the sample covariance matrix. In this paper we consider
the situation where the data variance is already partially explained by other
factors, e.g. covariates of interest, or temporal correlations leaving some
residual variance. We decompose the residual variance into its components
through a generalized eigenvalue problem, which we call residual component
analysis (RCA). We show that canonical covariates analysis (CCA) is a special
case of our algorithm and explore a range of new algorithms that arise from the
framework. We illustrate the ideas on a gene expression time series data set
and the recovery of human pose from silhouette.
