  This article addresses the problem of Ultra Reliable Low Latency
Communications (URLLC) in wireless networks, a framework with particularly
stringent constraints imposed by many Internet of Things (IoT) applications
from diverse sectors. We propose a novel Deep Reinforcement Learning (DRL)
scheduling algorithm, named NOMA-PPO, to solve the Non-Orthogonal Multiple
Access (NOMA) uplink URLLC scheduling problem involving strict deadlines. The
challenge of addressing uplink URLLC requirements in NOMA systems is related to
the combinatorial complexity of the action space due to the possibility to
schedule multiple devices, and to the partial observability constraint that we
impose to our algorithm in order to meet the IoT communication constraints and
be scalable. Our approach involves 1) formulating the NOMA-URLLC problem as a
Partially Observable Markov Decision Process (POMDP) and the introduction of an
agent state, serving as a sufficient statistic of past observations and
actions, enabling a transformation of the POMDP into a Markov Decision Process
(MDP); 2) adapting the Proximal Policy Optimization (PPO) algorithm to handle
the combinatorial action space; 3) incorporating prior knowledge into the
learning agent with the introduction of a Bayesian policy. Numerical results
reveal that not only does our approach outperform traditional multiple access
protocols and DRL benchmarks on 3GPP scenarios, but also proves to be robust
under various channel and traffic configurations, efficiently exploiting
inherent time correlations.
