  This paper focuses on the information freshness of finite-state Markov
sources, using the uncertainty of information (UoI) as the performance metric.
Measured by Shannon's entropy, UoI can capture not only the transition dynamics
of the Markov source but also the different evolutions of information quality
caused by the different values of the last observation. We consider an
information update system with M finite-state Markov sources transmitting
information to a remote monitor via m communication channels. Our goal is to
explore the optimal scheduling policy to minimize the sum-UoI of the Markov
sources. The problem is formulated as a restless multi-armed bandit (RMAB). We
relax the RMAB and then decouple the relaxed problem into M single bandit
problems. Analyzing the single bandit problem provides useful properties with
which the relaxed problem reduces to maximizing a concave and piecewise linear
function, allowing us to develop a gradient method to solve the relaxed problem
and obtain its optimal policy. By rounding up the optimal policy for the
relaxed problem, we obtain an index policy for the original RMAB problem.
Notably, the proposed index policy is universal in the sense that it applies to
general RMABs with bounded cost functions.
