  This letter investigates the problem of anti-jamming communications in
dynamic and unknown environment through on-line learning. Different from
existing studies which need to know (estimate) the jamming patterns and
parameters, we use the spectrum waterfall, i.e., the raw spectrum environment,
directly. Firstly, to cope with the challenge of infinite state of raw spectrum
information, a deep anti-jamming Q-network is constructed. Then, a deep
anti-jamming reinforcement learning algorithm is proposed to obtain the optimal
anti-jamming strategies. Finally, simulation results validate the the proposed
approach. The proposed approach is relying only on the local observed
information and does not need to estimate the jamming patterns and parameters,
which implies that it can be widely used various anti-jamming scenarios.
