  A simultaneous transmitting and reflecting reconfigurable intelligent surface
(STAR-RIS) assisted multi-user downlink multiple-input single-output (MISO)
communication system is investigated. In contrast to the existing ideal
STAR-RIS model assuming an independent transmission and reflection phase-shift
control, a practical coupled phase-shift model is considered. Then, a joint
active and passive beamforming optimization problem is formulated for
minimizing the long-term transmission power consumption, subject to the coupled
phase-shift constraint and the minimum data rate constraint. Despite the
coupled nature of the phase-shift model, the formulated problem is solved by
invoking a hybrid continuous and discrete phase-shift control policy. Inspired
by this observation, a pair of hybrid reinforcement learning (RL) algorithms,
namely the hybrid deep deterministic policy gradient (hybrid DDPG) algorithm
and the joint DDPG & deep-Q network (DDPG-DQN) based algorithm are proposed.
The hybrid DDPG algorithm controls the associated high-dimensional continuous
and discrete actions by relying on the hybrid action mapping. By contrast, the
joint DDPG-DQN algorithm constructs two Markov decision processes (MDPs)
relying on an inner and an outer environment, thereby amalgamating the two
agents to accomplish a joint hybrid control. Simulation results demonstrate
that the STAR-RIS has superiority over other conventional RISs in terms of its
energy consumption. Furthermore, both the proposed algorithms outperform the
baseline DDPG algorithm, and the joint DDPG-DQN algorithm achieves a superior
performance, albeit at an increased computational complexity.
