  The effective operation of time-critical Internet of things (IoT)
applications requires real-time reporting of fresh status information of
underlying physical processes. In this paper, a real-time IoT monitoring system
is considered, in which the IoT devices sample a physical process with a
sampling cost and send the status packet to a given destination with an
updating cost. This joint status sampling and updating process is designed to
minimize the average age of information (AoI) at the destination node under an
average energy cost constraint at each device. This is formulated as an
infinite horizon average cost constrained Markov decision process (CMDP) and
transformed into an unconstrained MDP using a Lagrangian method. For the single
IoT device case, the optimal policy for the CMDP is shown to be a randomized
mixture of two deterministic policies for the unconstrained MDP, which is of
threshold type. Then, a structure-aware optimal algorithm to obtain the optimal
policy of the CMDP is proposed and the impact of the wireless channel dynamics
is studied while demonstrating that channels having a larger mean channel gain
and less scattering can achieve better AoI performance. For the case of
multiple IoT devices, a low-complexity distributed suboptimal policy is
proposed with the updating control at the destination and the sampling control
at each device. Then, an online learning algorithm is developed to obtain this
policy, which can be implemented at each IoT device and requires only the local
knowledge and small signaling from the destination. The proposed learning
algorithm is shown to converge almost surely to the suboptimal policy.
Simulation results show the structural properties of the optimal policy for the
single IoT device case; and show that the proposed policy for multiple IoT
devices outperforms a zero-wait baseline policy, with average AoI reductions
reaching up to 33%.
