  The powerful learning ability of deep neural networks enables reinforcement
learning agents to learn competent control policies directly from continuous
environments. In theory, to achieve stable performance, neural networks assume
i.i.d. inputs, which unfortunately does no hold in the general reinforcement
learning paradigm where the training data is temporally correlated and
non-stationary. This issue may lead to the phenomenon of "catastrophic
interference" and the collapse in performance. In this paper, we present IQ,
i.e., interference-aware deep Q-learning, to mitigate catastrophic interference
in single-task deep reinforcement learning. Specifically, we resort to online
clustering to achieve on-the-fly context division, together with a multi-head
network and a knowledge distillation regularization term for preserving the
policy of learned contexts. Built upon deep Q networks, IQ consistently boosts
the stability and performance when compared to existing methods, verified with
extensive experiments on classic control and Atari tasks. The code is publicly
available at: https://github.com/Sweety-dm/Interference-aware-Deep-Q-learning.
