  In this paper, a novel joint energy and age of information (AoI) optimization
framework for IoT devices in a non-stationary environment is presented. In
particular, IoT devices that are distributed in the real-world are required to
efficiently utilize their computing resources so as to balance the freshness of
their data and their energy consumption. To optimize the performance of IoT
devices in such a dynamic setting, a novel lifelong reinforcement learning (RL)
solution that enables IoT devices to continuously adapt their policies to each
newly encountered environment is proposed. Given that IoT devices have limited
energy and computing resources, an unmanned aerial vehicle (UAV) is leveraged
to visit the IoT devices and update the policy of each device sequentially. As
such, the UAV is exploited as a mobile learning agent that can learn a shared
knowledge base with a feature base in its training phase, and feature sets of a
zero-shot learning method in its testing phase, to generalize between the
environments. To optimize the trajectory and flying velocity of the UAV, an
actor-critic network is leveraged so as to minimize the UAV energy consumption.
Simulation results show that the proposed lifelong RL solution can outperform
the state-of-art benchmarks by enhancing the balanced cost of IoT devices by
$8.3\%$ when incorporating warm-start policies for unseen environments. In
addition, our solution achieves up to $49.38\%$ reduction in terms of energy
consumption by the UAV in comparison to the random flying strategy.
