  We propose a deep reinforcement learning (DRL) approach for a full-duplex
(FD) transmission that predicts the phase shifts of the reconfigurable
intelligent surface (RIS), base station (BS) active beamformers, and the
transmit powers to maximize the weighted sum rate of uplink and downlink users.
Existing methods require channel state information (CSI) and residual
self-interference (SI) knowledge to calculate exact active beamformers or the
DRL rewards, which typically fail without CSI or residual SI. Especially for
time-varying channels, estimating and signaling CSI to the DRL agent is
required at each time step and is costly. We propose a two-stage DRL framework
with minimal signaling overhead to address this. The first stage uses the least
squares method to initiate learning by partially canceling the residual SI. The
second stage uses DRL to achieve performance comparable to existing CSI-based
methods without requiring the CSI or the exact residual SI. Further, the
proposed DRL framework for quantized RIS phase shifts reduces the signaling
from BS to the RISs using $32$ times fewer bits than the continuous version.
The quantized methods reduce action space, resulting in faster convergence and
$7.1\%$ and $22.28\%$ better UL and DL rates, respectively than the continuous
method.
