  In this paper, we generate a transmit power pool (PP) for Internet of things
(IoT) networks with semi-grant-free non-orthogonal multiple access (SGF-NOMA)
via multi-agent deep reinforcement learning (MA-DRL) to enable open loop power
control (PC). The PP is mapped with each resource block (RB) to achieve
distributed power control (DPC). We first formulate the resource allocation
problem as stochastic Markov game, and then solve it using two MA-DRL
algorithms, namely double deep Q network (DDQN) and Dueling DDQN. Each GF user
as an agent tries to find out the optimal transmit power level and RB to form
the desired PP. With the aid of dueling processes, the learning process can be
enhanced by evaluating the valuable state without considering the effect of
each action at each state. Therefore, DDQN is designed for communication
scenarios with a small-size action-state space, while Dueling DDQN is for a
large-size case. Moreover, to decrease the training time, we reduce the action
space by eliminating invalid actions. To control the interference and guarantee
the quality-of-service requirements of grant-based users, we determine the
optimal number of GF users for each sub-channel. We show that the PC approach
has a strong impact on data rates of both grant-based and GF users. We
demonstrate that the proposed algorithm is computationally scalable to
large-scale IoT networks and produce minimal signalling overhead. Our results
show that the proposed MA-Dueling DDQN based SGF-NOMA with DPC outperforms the
existing SGF-NOMA system and networks with pure GF protocols with 17.5\% and
22.2\% gain in terms of the system throughput, respectively. Finally, we show
that our proposed algorithm outperforms the conventional open loop PC
mechanism.
