  Assuming that one-step transition kernel of a discrete time, time-homogenous
Markov chain model is parameterized by a parameter $\theta\in \boldsymbol
\Theta$, we derive a recursive (in time) construction of confidence regions for
the unknown parameter of interest, say $\theta^*\in \boldsymbol \Theta$. It is
supposed that the observed data used in construction of the confidence regions
is generated by a Markov chain whose transition kernel corresponds to
$\theta^*$ . The key step in our construction is derivation of a recursive
scheme for an appropriate point estimator of $\theta^*$. To achieve this, we
start by what we call the base recursive point estimator, using which we design
a quasi-asymptotically linear recursive point estimator (a concept introduced
in this paper). For the latter estimator we prove its weak consistency and
asymptotic normality. The recursive construction of confidence regions is
needed not only for the purpose of speeding up the computation of the
successive confidence regions, but, primarily, for the ability to apply the
dynamic programming principle in the context of robust adaptive stochastic
control methodology.
