  Deep Reinforcement Learning (DRL) solutions are becoming pervasive at the
edge of the network as they enable autonomous decision-making in a dynamic
environment. However, to be able to adapt to the ever-changing environment, the
DRL solution implemented on an embedded device has to continue to occasionally
take exploratory actions even after initial convergence. In other words, the
device has to occasionally take random actions and update the value function,
i.e., re-train the Artificial Neural Network (ANN), to ensure its performance
remains optimal. Unfortunately, embedded devices often lack processing power
and energy required to train the ANN. The energy aspect is particularly
challenging when the edge device is powered only by a means of Energy
Harvesting (EH). To overcome this problem, we propose a two-part algorithm in
which the DRL process is trained at the sink. Then the weights of the fully
trained underlying ANN are periodically transferred to the EH-powered embedded
device taking actions. Using an EH-powered sensor, real-world measurements
dataset, and optimizing for Age of Information (AoI) metric, we demonstrate
that such a DRL solution can operate without any degradation in the
performance, with only a few ANN updates per day.
