  Beamforming is one of the key techniques in millimeter wave (mmWave)
multi-input multi-output (MIMO) communications. Designing appropriate
beamforming not only improves the quality and strength of the received signal,
but also can help reduce the interference, consequently enhancing the data
rate. In this paper, we propose a distributed multi-agent double deep
Q-learning algorithm for beamforming in mmWave MIMO networks, where multiple
base stations (BSs) can automatically and dynamically adjust their beams to
serve multiple highly-mobile user equipments (UEs). In the analysis, largest
received power association criterion is considered for UEs, and a realistic
channel model is taken into account. Simulation results demonstrate that the
proposed learning-based algorithm can achieve comparable performance with
respect to exhaustive search while operating at much lower complexity.
