  Though convolutional neural networks are widely used in different tasks, lack
of generalization capability in the absence of sufficient and representative
data is one of the challenges that hinder their practical application. In this
paper, we propose a simple, effective, and plug-and-play training strategy
named Knowledge Distillation for Domain Generalization (KDDG) which is built
upon a knowledge distillation framework with the gradient filter as a novel
regularization term. We find that both the ``richer dark knowledge" from the
teacher network, as well as the gradient filter we proposed, can reduce the
difficulty of learning the mapping which further improves the generalization
ability of the model. We also conduct experiments extensively to show that our
framework can significantly improve the generalization capability of deep
neural networks in different tasks including image classification,
segmentation, reinforcement learning by comparing our method with existing
state-of-the-art domain generalization techniques. Last but not the least, we
propose to adopt two metrics to analyze our proposed method in order to better
understand how our proposed method benefits the generalization capability of
deep neural networks.
