  We consider a dynamic multichannel access problem, where multiple correlated
channels follow an unknown joint Markov model. A user at each time slot selects
a channel to transmit data and receives a reward based on the success or
failure of the transmission. The objective is to find a policy that maximizes
the expected long-term reward. The problem is formulated as a partially
observable Markov decision process (POMDP) with unknown system dynamics. To
overcome the challenges of unknown system dynamics as well as prohibitive
computation, we apply the concept of reinforcement learning and implement a
Deep Q-Network (DQN) that can deal with large state space without any prior
knowledge of the system dynamics. We provide an analytical study on the optimal
policy for fixed-pattern channel switching with known system dynamics and show
through simulations that DQN can achieve the same optimal performance without
knowing the system statistics. We compare the performance of DQN with a Myopic
policy and a Whittle Index-based heuristic through both simulations as well as
real-data trace and show that DQN achieves near-optimal performance in more
complex situations. Finally, we propose an adaptive DQN approach with the
capability to adapt its learning in time-varying, dynamic scenarios.
