  This paper considers utility optimal power control for energy harvesting
wireless devices with a finite capacity battery. The distribution information
of the underlying wireless environment and harvestable energy is unknown and
only outdated system state information is known at the device controller. This
scenario shares similarity with Lyapunov opportunistic optimization and online
learning but is different from both. By a novel combination of Zinkevich's
online gradient learning technique and the drift-plus-penalty technique from
Lyapunov opportunistic optimization, this paper proposes a learning-aided
algorithm that achieves utility within $O(\epsilon)$ of the optimal, for any
desired $\epsilon>0$, by using a battery with an $O(1/\epsilon)$ capacity. The
proposed algorithm has low complexity and makes power investment decisions
based on system history, without requiring knowledge of the system state or its
probability distribution.
