  Markov state models (MSMs) have been successful in computing metastable
states, slow relaxation timescales and associated structural changes, and
stationary or kinetic experimental observables of complex molecules from large
amounts of molecular dynamics simulation data. However, MSMs approximate the
true dynamics by assuming a Markov chain on a clusters discretization of the
state space. This approximation is difficult to make for high-dimensional
biomolecular systems, and the quality and reproducibility of MSMs has therefore
been limited. Here, we discard the assumption that dynamics are Markovian on
the discrete clusters. Instead, we only assume that the full phase- space
molecular dynamics is Markovian, and a projection of this full dynamics is
observed on the discrete states, leading to the concept of Projected Markov
Models (PMMs). Robust estimation methods for PMMs are not yet available, but we
derive a practically feasible approximation via Hidden Markov Models (HMMs). It
is shown how various molecular observables of interest that are often computed
from MSMs can be computed from HMMs / PMMs. The new framework is applicable to
both, simulation and single-molecule experimental data. We demonstrate its
versatility by applications to educative model systems, an 1 ms Anton MD
simulation of the BPTI protein, and an optical tweezer force probe trajectory
of an RNA hairpin.
