  The time average expected age of information (AoI) is studied for status
updates sent over an error-prone channel from an energy-harvesting transmitter
with a finite-capacity battery. Energy cost of sensing new status updates is
taken into account as well as the transmission energy cost better capturing
practical systems. The optimal scheduling policy is first studied under the
hybrid automatic repeat request (HARQ) protocol when the channel and energy
harvesting statistics are known, and the existence of a threshold-based optimal
policy is shown. For the case of unknown environments, average-cost
reinforcement-learning algorithms are proposed that learn the system parameters
and the status update policy in real-time. The effectiveness of the proposed
methods is demonstrated through numerical results.
