  The problem of rested and restless multi-armed bandits with constrained
availability of arms is considered. The states of arms evolve in Markovian
manner and the exact states are hidden from the decision maker. First, some
structural results on value functions are claimed. Following these results, the
optimal policy turns out to be a \textit{threshold policy}. Further,
\textit{indexability} of rested bandits is established and index formula is
derived. The performance of index policy is illustrated and compared with
myopic policy using numerical examples.
