  We adopt an optimal-control framework for addressing the undiscounted
infinite-horizon discrete-time restless $N$-armed bandit problem. Unlike most
studies that rely on constructing policies based on the relaxed single-armed
Markov Decision Process (MDP), we propose relaxing the entire bandit MDP as an
optimal-control problem through the certainty equivalence control principle.
Our main contribution is demonstrating that the reachability of an optimal
stationary state within the optimal-control problem is a sufficient condition
for the existence of an asymptotically optimal policy. Such a policy can be
devised using an "align and steer" strategy. This reachability assumption is
less stringent than any prior assumptions imposed on the arm-level MDP, notably
the unichain condition is no longer needed. Through numerical examples, we show
that employing model predictive control for steering generally results in
superior performance compared to other existing policies.
