  We describe an algorithm for computing the maximal invariant set for a Markov
chain with linear safety constraints on the distribution over states. We then
propose a Markov chain synthesis method that guarantees finite determination of
the maximal invariant set. Although this problem is bilinear in the general
case, we are able to optimize the convergence rate to a desirable steady-state
distribution over reversible Markov chains by solving a Semidefinite Program
(SDP), which promotes efficient computation of the maximal invariant set. We
then demonstrate this approach with a decentralized swarm guidance application
subject to density upper bounds.
