  In this paper, we develop optimal policies for deciding when a wireless node
with radio frequency (RF) energy harvesting (EH) capabilities should try and
harvest ambient RF energy. While the idea of RF-EH is appealing, it is not
always beneficial to attempt to harvest energy; in environments where the
ambient energy is low, nodes could consume more energy being awake with their
harvesting circuits turned on than what they can extract from the ambient radio
signals; it is then better to enter a sleep mode until the ambient RF energy
increases. Towards this end, we consider a scenario with intermittent energy
arrivals and a wireless node that wakes up for a period of time (herein called
the time-slot) and harvests energy. If enough energy is harvested during the
time-slot, then the harvesting is successful and excess energy is stored;
however, if there does not exist enough energy the harvesting is unsuccessful
and energy is lost.
  We assume that the ambient energy level is constant during the time-slot, and
changes at slot boundaries. The energy level dynamics are described by a
two-state Gilbert-Elliott Markov chain model, where the state of the Markov
chain can only be observed during the harvesting action, and not when in sleep
mode. Two scenarios are studied under this model. In the first scenario, we
assume that we have knowledge of the transition probabilities of the Markov
chain and formulate the problem as a Partially Observable Markov Decision
Process (POMDP), where we find a threshold-based optimal policy. In the second
scenario, we assume that we don't have any knowledge about these parameters and
formulate the problem as a Bayesian adaptive POMDP; to reduce the complexity of
the computations we also propose a heuristic posterior sampling algorithm. The
performance of our approaches is demonstrated via numerical examples.
