  We present a novel analysis of the dynamics of tensor power iterations in the
overcomplete regime where the tensor CP rank is larger than the input
dimension. Finding the CP decomposition of an overcomplete tensor is NP-hard in
general. We consider the case where the tensor components are randomly drawn,
and show that the simple power iteration recovers the components with bounded
error under mild initialization conditions. We apply our analysis to
unsupervised learning of latent variable models, such as multi-view mixture
models and spherical Gaussian mixtures. Given the third order moment tensor, we
learn the parameters using tensor power iterations. We prove it can correctly
learn the model parameters when the number of hidden components $k$ is much
larger than the data dimension $d$, up to $k = o(d^{1.5})$. We initialize the
power iterations with data samples and prove its success under mild conditions
on the signal-to-noise ratio of the samples. Our analysis significantly expands
the class of latent variable models where spectral methods are applicable. Our
analysis also deals with noise in the input tensor leading to sample complexity
result in the application to learning latent variable models.
