  In this paper, the problem of dynamic spectrum sensing and aggregation is
investigated in a wireless network containing N correlated channels, where
these channels are occupied or vacant following an unknown joint 2-state Markov
model. At each time slot, a single cognitive user with certain bandwidth
requirement either stays idle or selects a segment comprising C (C < N)
contiguous channels to sense. Then, the vacant channels in the selected segment
will be aggregated for satisfying the user requirement. The user receives a
binary feedback signal indicating whether the transmission is successful or not
(i.e., ACK signal) after each transmission, and makes next decision based on
the sensing channel states. Here, we aim to find a policy that can maximize the
number of successful transmissions without interrupting the primary users
(PUs). The problem can be considered as a partially observable Markov decision
process (POMDP) due to without full observation of system environment. We
implement a Deep Q-Network (DQN) to address the challenge of unknown system
dynamics and computational expenses. The performance of DQN, Q-Learning, and
the Improvident Policy with known system dynamics is evaluated through
simulations. The simulation results show that DQN can achieve near-optimal
performance among different system scenarios only based on partial observations
and ACK signals.
