  We consider the restless bandits with general state space under partial
observability with two observational models: first, the state of each bandit is
not observable at all, and second, the state of each bandit is observable only
if it is chosen. We assume both models satisfy the restart property under which
we prove indexability of the models and propose the Whittle index policy as the
solution. For the first model, we derive a closed-form expression for the
Whittle index. For the second model, we propose an efficient algorithm to
compute the Whittle index by exploiting the qualitative properties of the
optimal policy. We present detailed numerical experiments for multiple
instances of machine maintenance problem. The result indicates that the Whittle
index policy outperforms myopic policy and can be close to optimal in different
setups.
