  Millimeter wave (mmWave) device to device (D2D) communication is highly
susceptible to obstacles due to severe penetration losses and requires almost a
line of sight (LOS) communication path. D2D channel condition is local to
devices/user equipments (UEs) and hence is \textit{not} directly visible to the
base station (BS). Thus quality of the D2D channel needs to be propagated to BS
by UEs which may incur some delay. Hence the solution provided by BS to UEs
using this gathered channel information might become less useful to establish
communication due to moving obstacles. These types of obstacles might not be
known in advance and hence may cause unpredictable fluctuations to the D2D
channel quality. Hence we seek to learn the D2D channels using the finite
horizon partially observable Markov decision process (POMDP) framework to model
the uncertainty in such kind of network environments with dynamic obstacles.
The objective is to minimize delay when channel quality deteriorates, by making
UEs choose locally the best possible decision between i) to continue on the
current relay link on which communication is taking place or ii) to switch to
another good relay by exploring other possible UEs in its locality. We derive
an optimal threshold policy which tells the UE to take appropriate decision
locally. Later, we give a simplified and easy to implement stationary threshold
policy which counts the number of successive acknowledgement failures, based on
which UE make appropriate decision locally. Through extensive simulation, we
demonstrate that our approach outperforms recent algorithms.
