  Owing to the unique advantages of low cost and controllability,
reconfigurable intelligent surface (RIS) is a promising candidate to address
the blockage issue in millimeter wave (mmWave) communication systems,
consequently has captured widespread attention in recent years. However, the
joint active beamforming and passive beamforming design is an arduous task due
to the high computational complexity and the dynamic changes of wireless
environment. In this paper, we consider a RIS-assisted multi-user
multiple-input single-output (MU-MISO) mmWave system and aim to develop a deep
reinforcement learning (DRL) based algorithm to jointly design active hybrid
beamformer at the base station (BS) side and passive beamformer at the RIS
side. By employing an advanced soft actor-critic (SAC) algorithm, we propose a
maximum entropy based DRL algorithm, which can explore more stochastic policies
than deterministic policy, to design active analog precoder and passive
beamformer simultaneously. Then, the digital precoder is determined by minimum
mean square error (MMSE) method. The experimental results demonstrate that our
proposed SAC algorithm can achieve better performance compared with
conventional optimization algorithm and DRL algorithm.
