  We consider a cognitive heterogeneous network (HetNet), in which multiple
pairs of secondary users adopt sensing-based approaches to coexist with a pair
of primary users on a certain spectrum band. Due to imperfect spectrum sensing,
secondary transmitters (STs) may cause interference to the primary receiver
(PR) and make it difficult for the PR to select a proper modulation and/or
coding scheme (MCS). To deal with this issue, we exploit deep reinforcement
learning (DRL) and propose an intelligent MCS selection algorithm for the
primary transmission. To reduce the system overhead caused by MCS switchings,
we further introduce a switching cost factor in the proposed algorithm.
Simulation results show that the primary transmission rate of the proposed
algorithm without the switching cost factor is 90 percent to 100 percent of the
optimal MCS selection scheme, which assumes that the interference from the STs
is perfectly known at the PR as prior information, and is 30 percent to 100
percent higher than those of the benchmark algorithms. Meanwhile, the proposed
algorithm with the switching cost factor can achieve a better balance between
the primary transmission rate and system overheads than both the optimal
algorithm and benchmark algorithms.
