  We consider a discrete-time Markov chain $\boldsymbol{\Phi}$ on a general
state-space ${\sf X}$, whose transition probabilities are parameterized by a
real-valued vector $\boldsymbol{\theta}$. Under the assumption that
$\boldsymbol{\Phi}$ is geometrically ergodic with corresponding stationary
distribution $\pi(\boldsymbol{\theta})$, we are interested in estimating the
gradient $\nabla \alpha(\boldsymbol{\theta})$ of the steady-state expectation
$$\alpha(\boldsymbol{\theta}) = \pi( \boldsymbol{\theta}) f.$$
  To this end, we first give sufficient conditions for the differentiability of
$\alpha(\boldsymbol{\theta})$ and for the calculation of its gradient via a
sequence of finite horizon expectations. We then propose two different
likelihood ratio estimators and analyze their limiting behavior.
