  Factor analysis provides linear factors that describe relationships between
individual variables of a data set. We extend this classical formulation into
linear factors that describe relationships between groups of variables, where
each group represents either a set of related variables or a data set. The
model also naturally extends canonical correlation analysis to more than two
sets, in a way that is more flexible than previous extensions. Our solution is
formulated as variational inference of a latent variable model with structural
sparsity, and it consists of two hierarchical levels: The higher level models
the relationships between the groups, whereas the lower models the observed
variables given the higher level. We show that the resulting solution solves
the group factor analysis problem accurately, outperforming alternative factor
analysis based solutions as well as more straightforward implementations of
group factor analysis. The method is demonstrated on two life science data
sets, one on brain activation and the other on systems biology, illustrating
its applicability to the analysis of different types of high-dimensional data
sources.
