  Caching has been regarded as a promising technique to alleviate energy
consumption of sensors in Internet of Things (IoT) networks by responding to
users' requests with the data packets stored in the edge caching node (ECN).
For real-time applications in caching enabled IoT networks, it is essential to
develop dynamic status update strategies to strike a balance between the
information freshness experienced by users and energy consumed by the sensor,
which, however, is not well addressed. In this paper, we first depict the
evolution of information freshness, in terms of age of information (AoI), at
each user. Then, we formulate a dynamic status update optimization problem to
minimize the expectation of a long term accumulative cost, which jointly
considers the users' AoI and sensor's energy consumption. To solve this
problem, a Markov Decision Process (MDP) is formulated to cast the status
updating procedure, and a model-free reinforcement learning algorithm is
proposed, with which the challenge brought by the unknown of the formulated
MDP's dynamics can be addressed. Finally, simulations are conducted to validate
the convergence of our proposed algorithm and its effectiveness compared with
the zero-wait baseline policy.
