  Deep deraining networks consistently encounter substantial generalization
issues when deployed in real-world applications, although they are successful
in laboratory benchmarks. A prevailing perspective in deep learning encourages
using highly complex data for training, with the expectation that richer image
background content will facilitate overcoming the generalization problem.
However, through comprehensive and systematic experimentation, we discover that
this strategy does not enhance the generalization capability of these networks.
On the contrary, it exacerbates the tendency of networks to overfit specific
degradations. Our experiments reveal that better generalization in a deraining
network can be achieved by simplifying the complexity of the training
background images. This is because that the networks are ``slacking off''
during training, that is, learning the least complex elements in the image
background and degradation to minimize training loss. When the background
images are less complex than the rain streaks, the network will prioritize the
background reconstruction, thereby suppressing overfitting the rain patterns
and leading to improved generalization performance. Our research offers a
valuable perspective and methodology for better understanding the
generalization problem in low-level vision tasks and displays promising
potential for practical application.
