  In this paper, we consider the multi-user scheduling problem in millimeter
wave (mmWave) video streaming networks, which comprise a streaming server and
several users, each requesting a video stream with a different resolution. The
main objective is to optimize the long-term average quality of experience (QoE)
for all users. We tackle this problem by considering the physical layer
characteristics of the mmWave network, including the beam alignment overhead
due to pencil-beams. To develop an efficient scheduling policy, we leverage the
contextual multi-armed bandit (MAB) models to propose a beam alignment overhead
and buffer predictive streaming solution, dubbed B2P-Stream. The proposed
B2P-Stream algorithm optimally balances the trade-off between the overhead and
users' buffer levels and improves the QoE by reducing the beam alignment
overhead for users of higher resolutions. We also provide a theoretical
guarantee for our proposed method and prove that it guarantees a sub-linear
regret bound. Finally, we examine our proposed framework through extensive
simulations. We provide a detailed comparison of the B2P-Stream against
uniformly random and Round-robin (RR) policies and show that it outperforms
both of them in providing a better QoE and fairness. We also analyze the
scalability and robustness of the B2P-Stream algorithm with different network
configurations.
