  In this paper, dynamic non-cooperative coexistence between a cognitive pulsed
radar and a nearby communications system is addressed by applying nonlinear
value function approximation via deep reinforcement learning (Deep RL) to
develop a policy for optimal radar performance. The radar learns to vary the
bandwidth and center frequency of its linear frequency modulated (LFM)
waveforms to mitigate mutual interference with other systems and improve target
detection performance while also maintaining sufficient utilization of the
available frequency bands required for a fine range resolution. We demonstrate
that our approach, based on the Deep Q-Learning (DQL) algorithm, enhances
important radar metrics, including SINR and bandwidth utilization, more
effectively than policy iteration or sense-and-avoid (SAA) approaches in a
variety of realistic coexistence environments. We also extend the DQL-based
approach to incorporate Double Q-learning and a recurrent neural network to
form a Double Deep Recurrent Q-Network (DDRQN). We demonstrate the DDRQN
results in favorable performance and stability compared to DQL and policy
iteration. Finally, we demonstrate the practicality of our proposed approach
through a discussion of experiments performed on a software defined radar
(SDRadar) prototype system. Our experimental results indicate that the proposed
Deep RL approach significantly improves radar detection performance in
congested spectral environments when compared to policy iteration and SAA.
