  Wireless underground sensor networks play an important role in underground
sensing such as climate-smart agriculture and underground infrastructure
monitoring. Existing works consider a static underground environment, which is
not practical since the dielectric parameters of soil change frequently due to
precipitation and harsh weather. This challenge cannot be ignored in real
implementation due to the drastic change of wireless underground channel. In
this paper, we study the effect of dynamic underground environment on wireless
communications for sensor networks. We use the real data collected by in-situ
sensors to train a Hidden Markov Model. Then, by using reinforcement learning,
we derive the optimal transmission policies for underground sensors to
efficiently use their energy and reduce the number of dropped and
unsuccessfully transmitted packets. Through simulations using real data, we
find that the developed algorithm can reduce the packet loss and transmit the
sensed data in a timely manner.
