  Reversibility is a key concept in Markov models and Master-equation models of
molecular kinetics. The analysis and interpretation of the transition matrix
encoding the kinetic properties of the model relies heavily on the
reversibility property. The estimation of a reversible transition matrix from
simulation data is therefore crucial to the successful application of the
previously developed theory. In this work we discuss methods for the maximum
likelihood estimation of transition matrices from finite simulation data and
present a new algorithm for the estimation if reversibility with respect to a
given stationary vector is desired. We also develop new methods for the
Bayesian posterior inference of reversible transition matrices with and without
given stationary vector taking into account the need for a suitable prior
distribution preserving the meta- stable features of the observed process
during posterior inference. All algorithms here are implemented in the PyEMMA
software - http://pyemma.org - as of version 2.0.
