  A novel reinforcement learning algorithm is introduced for multiarmed
restless bandits with average reward, using the paradigms of Q-learning and
Whittle index. Specifically, we leverage the structure of the Whittle index
policy to reduce the search space of Q-learning, resulting in major
computational gains. Rigorous convergence analysis is provided, supported by
numerical experiments. The numerical experiments show excellent empirical
performance of the proposed scheme.
