  We consider a Markov control model in discrete time with countable both state
space and action space. Using the value function of a suitable long-run average
reward problem, we study various reachability/controllability problems. First,
we characterize the domain of attraction and escape set of the system, and a
generalization called $p$-domain of attraction, using the aforementioned value
function. Next, we solve the problem of maximizing the probability of reaching
a set $A$ while avoiding a set $B$. Finally, we consider a constrained version
of the previous problem where we ask for the probability of reaching the set
$B$ to be bounded. In the finite case, we use linear programming formulations
to solve these problems. Finally, we apply our results to a example of an
object that navigates under stochastic influence.
