  Internet of Things (IoT) devices can apply mobile-edge computing (MEC) and
energy harvesting (EH) to provide the satisfactory quality of experiences for
computation intensive applications and prolong the battery lifetime. In this
article, we investigate the computation offloading for IoT devices with energy
harvesting in wireless networks with multiple MEC devices such as base stations
and access points, each with different computation resource and radio
communication capability. We propose a reinforcement learning based computation
offloading framework for an IoT device to choose the MEC device and determine
the offloading rate according to the current battery level, the previous radio
bandwidth to each MEC device and the predicted amount of the harvested energy.
A "hotbooting" Q-learning based computation offloading scheme is proposed for
an IoT device to achieve the optimal offloading performance without being aware
of the MEC model, the energy consumption and computation latency model. We also
propose a fast deep Q-network (DQN) based offloading scheme, which combines the
deep learning and hotbooting techniques to accelerate the learning speed of
Q-learning. We show that the proposed schemes can achieve the optimal
offloading policy after sufficiently long learning time and provide their
performance bounds under two typical MEC scenarios. Simulations are performed
for IoT devices that use wireless power transfer to capture the ambient
radio-frequency signals to charge the IoT batteries. Simulation results show
that the fast DQN-based offloading scheme reduces the energy consumption,
decreases the computation delay and the task drop ratio, and increases the
utility of the IoT device in dynamic MEC, compared with the benchmark
Q-learning based offloading.
