  This paper aims to understand and improve the utility of the dropout
operation from the perspective of game-theoretic interactions. We prove that
dropout can suppress the strength of interactions between input variables of
deep neural networks (DNNs). The theoretic proof is also verified by various
experiments. Furthermore, we find that such interactions were strongly related
to the over-fitting problem in deep learning. Thus, the utility of dropout can
be regarded as decreasing interactions to alleviate the significance of
over-fitting. Based on this understanding, we propose an interaction loss to
further improve the utility of dropout. Experimental results have shown that
the interaction loss can effectively improve the utility of dropout and boost
the performance of DNNs.
