  Blockage due to obstacles in millimeter wave (mmWave) device to device (D2D)
communication is a prominent problem due to their severe penetration losses.
Potential user equipments (UEs) in vicinity of the source UE must be explored
in order to select a new relay when the current link gets blocked. However,
dynamic obstacles are not known in advance and thus may cause unpredictable
fluctuations to D2D channel quality causing newly selected relay link also to
be susceptible to blockage. This might cause frequent relay switching leading
to call drops and high energy consumption. We have proposed the idea of
reducing frequency in relay exploration and switching and thus average
end-to-end delay (in seconds) at the expense of additional exploration time
units (few milliseconds) during beam alignment. We seek to learn the
uncertainty in D2D link qualities by modeling the problem as finite horizon
partially observable Markov decision process (POMDP) framework locally at each
UE. We have derived an optimal threshold policy which maps the state to set of
actions. We then give a simplified and easy to implement stationary threshold
policy which counts the number of successive acknowledgment successes/failures
for making decisions of selecting or not selecting a given relay locally.
Through extensive simulation, we validate our theoretical findings and
demonstrate that our approach captures the trade-off between average
exploration time and average end-to-end (E2E) delay in presence of dynamic
obstacles.
