  For a multi-cell, multi-user, cellular network downlink sum-rate maximization
through power allocation is a nonconvex and NP-hard optimization problem. In
this paper, we present an effective approach to solving this problem through
single- and multi-agent actor-critic deep reinforcement learning (DRL).
Specifically, we use finite-horizon trust region optimization. Through
extensive simulations, we show that we can simultaneously achieve higher
spectral efficiency than state-of-the-art optimization algorithms like weighted
minimum mean-squared error (WMMSE) and fractional programming (FP), while
offering execution times more than two orders of magnitude faster than these
approaches. Additionally, the proposed trust region methods demonstrate
superior performance and convergence properties than the Advantage Actor-Critic
(A2C) DRL algorithm. In contrast to prior approaches, the proposed
decentralized DRL approaches allow for distributed optimization with limited
CSI and controllable information exchange between BSs while offering
competitive performance and reduced training times.
