  The gain of free energy upon unmixing is determined via application of Markov
state modeling (MSM), using an Ising model with a fixed number of up- and
down-spins. MSM yields reasonable estimates of the free energies. However, a
closer look reveals significant differences which point to residual
non-Markovian effects. These non-Markovian effects are rather unexpected since
the typical criteria to study the quality of Markovianity indicate complete
Markovian behavior. We identify the sparse connectivity between different
Markov states as a likely reason for the observed bias. By studying a simple
five state model system we can analytically elucidate different sources of the
bias and thus explain the different deviations that were observed for the Ising
model. Based on this insight we can modify the determination of the count
matrix in the MSM approach. In this way, the estimation of the free energy is
significantly improved.
