  This paper presents a novel and effective deep reinforcement learning
(DRL)-based approach to addressing joint resource management (JRM) in a
practical multi-carrier non-orthogonal multiple access (MC-NOMA) system, where
hardware sensitivity and imperfect successive interference cancellation (SIC)
are considered. We first formulate the JRM problem to maximize the weighted-sum
system throughput. Then, the JRM problem is decoupled into two iterative
subtasks: subcarrier assignment (SA, including user grouping) and power
allocation (PA). Each subtask is a sequential decision process. Invoking a deep
deterministic policy gradient algorithm, our proposed DRL-based JRM (DRL-JRM)
approach jointly performs the two subtasks, where the optimization objective
and constraints of the subtasks are addressed by a new joint reward and
internal reward mechanism. A multi-agent structure and a convolutional neural
network are adopted to reduce the complexity of the PA subtask. We also tailor
the neural network structure for the stability and convergence of DRL-JRM.
Corroborated by extensive experiments, the proposed DRL-JRM scheme is superior
to existing alternatives in terms of system throughput and resistance to
interference, especially in the presence of many users and strong inter-cell
interference. DRL-JRM can flexibly meet individual service requirements of
users.
