  We propose Re-parameterized Refocusing Convolution (RefConv) as a replacement
for regular convolutional layers, which is a plug-and-play module to improve
the performance without any inference costs. Specifically, given a pre-trained
model, RefConv applies a trainable Refocusing Transformation to the basis
kernels inherited from the pre-trained model to establish connections among the
parameters. For example, a depth-wise RefConv can relate the parameters of a
specific channel of convolution kernel to the parameters of the other kernel,
i.e., make them refocus on the other parts of the model they have never
attended to, rather than focus on the input features only. From another
perspective, RefConv augments the priors of existing model structures by
utilizing the representations encoded in the pre-trained parameters as the
priors and refocusing on them to learn novel representations, thus further
enhancing the representational capacity of the pre-trained model. Experimental
results validated that RefConv can improve multiple CNN-based models by a clear
margin on image classification (up to 1.47% higher top-1 accuracy on ImageNet),
object detection and semantic segmentation without introducing any extra
inference costs or altering the original model structure. Further studies
demonstrated that RefConv can reduce the redundancy of channels and smooth the
loss landscape, which explains its effectiveness.
