  Latent variable models with hidden binary units appear in various
applications. Learning such models, in particular in the presence of noise, is
a challenging computational problem. In this paper we propose a novel spectral
approach to this problem, based on the eigenvectors of both the second order
moment matrix and third order moment tensor of the observed data. We prove that
under mild non-degeneracy conditions, our method consistently estimates the
model parameters at the optimal parametric rate. Our tensor-based method
generalizes previous orthogonal tensor decomposition approaches, where the
hidden units were assumed to be either statistically independent or mutually
exclusive. We illustrate the consistency of our method on simulated data and
demonstrate its usefulness in learning a common model for population mixtures
in genetics.
