  Whittle index policy is a powerful tool to obtain asymptotically optimal
solutions for the notoriously intractable problem of restless bandits. However,
finding the Whittle indices remains a difficult problem for many practical
restless bandits with convoluted transition kernels. This paper proposes
NeurWIN, a neural Whittle index network that seeks to learn the Whittle indices
for any restless bandits by leveraging mathematical properties of the Whittle
indices. We show that a neural network that produces the Whittle index is also
one that produces the optimal control for a set of Markov decision problems.
This property motivates using deep reinforcement learning for the training of
NeurWIN. We demonstrate the utility of NeurWIN by evaluating its performance
for three recently studied restless bandit problems. Our experiment results
show that the performance of NeurWIN is significantly better than other RL
algorithms.
