  Importance sampling is a variance reduction technique for efficient
estimation of rare-event probabilities by Monte Carlo. In standard importance
sampling schemes, the system is simulated using an a priori fixed change of
measure suggested by a large deviation lower bound analysis. Recent work,
however, has suggested that such schemes do not work well in many situations.
In this paper we consider dynamic importance sampling in the setting of
uniformly recurrent Markov chains. By ``dynamic'' we mean that in the course of
a single simulation, the change of measure can depend on the outcome of the
simulation up till that time. Based on a control-theoretic approach to large
deviations, the existence of asymptotically optimal dynamic schemes is
demonstrated in great generality. The implementation of the dynamic schemes is
carried out with the help of a limiting Bellman equation. Numerical examples
are presented to contrast the dynamic and standard schemes.
