  Recently there has been substantial interest in spectral methods for learning
dynamical systems. These methods are popular since they often offer a good
tradeoff between computational and statistical efficiency. Unfortunately, they
can be difficult to use and extend in practice: e.g., they can make it
difficult to incorporate prior information such as sparsity or structure. To
address this problem, we present a new view of dynamical system learning: we
show how to learn dynamical systems by solving a sequence of ordinary
supervised learning problems, thereby allowing users to incorporate prior
knowledge via standard techniques such as L1 regularization. Many existing
spectral methods are special cases of this new framework, using linear
regression as the supervised learner. We demonstrate the effectiveness of our
framework by showing examples where nonlinear regression or lasso let us learn
better state representations than plain linear regression does; the correctness
of these instances follows directly from our general analysis.
