  For Industrial Wireless Sensor Networks, it is essential to reliably sense
and deliver the environmental data on time to avoid system malfunction. While
energy harvesting is a promising technique to extend the lifetime of sensor
nodes, it also brings new challenges for system reliability due to the
stochastic nature of the harvested energy. In this paper, we investigate the
optimal energy management policy to minimize the weighted packet loss rate
under delay constraint, where the packet loss rate considers the lost packets
both during the sensing and delivering processes. We show that the above energy
management problem can be modeled as an infinite horizon average reward
constraint Markov decision problem. In order to address the well-known curse of
dimensionality problem and facilitate distributed implementation, we utilize
the linear value approximation technique. Moreover, we apply stochastic online
learning with post-decision state to deal with the lack of knowledge of the
underlying stochastic processes. A distributed energy allocation algorithm with
water-filling structure and a scheduling algorithm by auction mechanism are
obtained. Experimental results show that the proposed algorithm achieves nearly
the same performance as the optimal offline value iteration algorithm while
requiring much less computation complexity and signaling overhead, and
outperforms various existing baseline algorithms.
