  Perturbation analysis of Markov chains provides bounds on the effect that a
change in a Markov transition matrix has on the corresponding stationary
distribution. This paper compares and analyzes bounds found in the literature
for finite and denumerable Markov chains and introduces new bounds based on
series expansions. We discuss a series of examples to illustrate the
applicability and numerical efficiency of the various bounds. Specifically, we
address the question on how the bounds developed for finite Markov chains
behave as the size of the system grows. In addition, we provide for the first
time an analysis of the relative error of these bounds. For the case of a
scaled perturbation we show that perturbation bounds can be used to analyze
stability of a stable Markov chain with respect to perturbation with an
unstable chain.
