  In this paper, we propose Selective Output Smoothing Regularization, a novel
regularization method for training the Convolutional Neural Networks (CNNs).
Inspired by the diverse effects on training from different samples, Selective
Output Smoothing Regularization improves the performance by encouraging the
model to produce equal logits on incorrect classes when dealing with samples
that the model classifies correctly and over-confidently. This plug-and-play
regularization method can be conveniently incorporated into almost any
CNN-based project without extra hassle. Extensive experiments have shown that
Selective Output Smoothing Regularization consistently achieves significant
improvement in image classification benchmarks, such as CIFAR-100, Tiny
ImageNet, ImageNet, and CUB-200-2011. Particularly, our method obtains 77.30%
accuracy on ImageNet with ResNet-50, which gains 1.1% than baseline (76.2%). We
also empirically demonstrate the ability of our method to make further
improvements when combining with other widely used regularization techniques.
On Pascal detection, using the SOSR-trained ImageNet classifier as the
pretrained model leads to better detection performances.
