  Reconfigurable intelligent surface (RIS) is considered as an extraordinarily
promising technology to solve the blockage problem of millimeter wave (mmWave)
communications owing to its capable of establishing a reconfigurable wireless
propagation. In this paper, we focus on a RIS-assisted mmWave communication
network consisting of multiple base stations (BSs) serving a set of user
equipments (UEs). Considering the BS-RIS-UE association problem which
determines that the RIS should assist which BS and UEs, we joint optimize
BS-RIS-UE association and passive beamforming at RIS to maximize the sum-rate
of the system. To solve this intractable non-convex problem, we propose a soft
actor-critic (SAC) deep reinforcement learning (DRL)-based joint beamforming
and BS-RIS-UE association design algorithm, which can learn the best policy by
interacting with the environment using less prior information and avoid falling
into the local optimal solution by incorporating with the maximization of
policy information entropy. The simulation results demonstrate that the
proposed SAC-DRL algorithm can achieve significant performance gains compared
with benchmark schemes.
