  This paper is in two parts. In the first part of this paper, formulae for
f-divergence for a continuous Markov process are reviewed. Applications of
these formulae are described for the problems of stochastic stability, second
law of thermodynamics, and non-equilibrium extensions thereof. The first part
sets the stage for considering the f-divergence for hidden Markov processes
which is the focus of the second part. For hidden Markov processes, again based
on the use of f-divergence, analyses of filter stability and stochastic
thermodynamics are described. The latter is used to illustrate the Maxwell
demon for an over-damped Langevin model with white noise type observations. The
expository nature of the paper combined with consistent formalism for both
Markov and hidden Markov processes is written to be useful for researchers
working on related themes in disparate areas.
