  To accommodate the explosive wireless traffics, massive multiple-input
multiple-output (MIMO) is regarded as one of the key enabling technologies for
next-generation communication systems. In massive MIMO cellular networks,
coordinated beamforming (CBF), which jointly designs the beamformers of
multiple base stations (BSs), is an efficient method to enhance the network
performance. In this paper, we investigate the sum rate maximization problem in
a massive MIMO mobile cellular network, where in each cell a multi-antenna BS
serves multiple mobile users simultaneously via downlink beamforming. Although
existing optimization-based CBF algorithms can provide near-optimal solutions,
they require realtime and global channel state information (CSI), in addition
to their high computation complexity. It is almost impossible to apply them in
practical wireless networks, especially highly dynamic mobile cellular
networks. Motivated by this, we propose a deep reinforcement learning based
distributed dynamic coordinated beamforming (DDCBF) framework, which enables
each BS to determine the beamformers with only local CSI and some historical
information from other BSs.Besides, the beamformers can be calculated with a
considerably lower computational complexity by exploiting neural networks and
expert knowledge, i.e., a solution structure observed from the iterative
procedure of the weighted minimum mean square error (WMMSE) algorithm.
Moreover, we provide extensive numerical simulations to validate the
effectiveness of the proposed DRL-based approach. With lower computational
complexity and less required information, the results show that the proposed
approach can achieve comparable performance to the centralized iterative
optimization algorithms.
