  Transfer operators such as Perron-Frobenius or Koopman operator play a key
role in modeling and analysis of complex dynamical systems, which allow linear
representations of nonlinear dynamics by transforming the original state
variables to feature spaces. However, it remains challenging to identify the
optimal low-dimensional feature mappings from data. The variational approach
for Markov processes (VAMP) provides a comprehensive framework for the
evaluation and optimization of feature mappings based on the variational
estimation of modeling errors, but it still suffers from a flawed assumption on
the transfer operator and therefore sometimes fails to capture the essential
structure of system dynamics. In this paper, we develop a powerful alternative
to VAMP, called kernel embedding based variational approach for dynamical
systems (KVAD). By using the distance measure of functions in the kernel
embedding space, KVAD effectively overcomes the theoretical and practical
limitations of VAMP. In addition, we develop a data-driven KVAD algorithm for
seeking the ideal feature mapping within a subspace spanned by given basis
functions, and numerical experiments show that the proposed algorithm can
significantly improve the modeling accuracy compared to VAMP.
