  Recently, there has been a surge of interest in using spectral methods for
estimating latent variable models. However, it is usually assumed that the
distribution of the observations conditioned on the latent variables is either
discrete or belongs to a parametric family. In this paper, we study the
estimation of an $m$-state hidden Markov model (HMM) with only smoothness
assumptions, such as H\"olderian conditions, on the emission densities. By
leveraging some recent advances in continuous linear algebra and numerical
analysis, we develop a computationally efficient spectral algorithm for
learning nonparametric HMMs. Our technique is based on computing an SVD on
nonparametric estimates of density functions by viewing them as
\emph{continuous matrices}. We derive sample complexity bounds via
concentration results for nonparametric density estimation and novel
perturbation theory results for continuous matrices. We implement our method
using Chebyshev polynomial approximations. Our method is competitive with other
baselines on synthetic and real problems and is also very computationally
efficient.
