  Deep learning offers state of the art solutions for image recognition.
However, deep models are vulnerable to adversarial perturbations in images that
are subtle but significantly change the model's prediction. In a white-box
attack, these perturbations are generally learned for deep models that operate
on RGB images and, hence, the perturbations are equally distributed in the RGB
color space. In this paper, we show that the adversarial perturbations prevail
in the Y-channel of the YCbCr space. Our finding is motivated from the fact
that the human vision and deep models are more responsive to shape and texture
rather than color. Based on our finding, we propose a defense against
adversarial images. Our defence, coined ResUpNet, removes perturbations only
from the Y-channel by exploiting ResNet features in an upsampling framework
without the need for a bottleneck. At the final stage, the untouched
CbCr-channels are combined with the refined Y-channel to restore the clean
image. Note that ResUpNet is model agnostic as it does not modify the DNN
structure. ResUpNet is trained end-to-end in Pytorch and the results are
compared to existing defence techniques in the input transformation category.
Our results show that our approach achieves the best balance between defence
against adversarial attacks such as FGSM, PGD and DDN and maintaining the
original accuracies of VGG-16, ResNet50 and DenseNet121 on clean images. We
perform another experiment to show that learning adversarial perturbations only
for the Y-channel results in higher fooling rates for the same perturbation
magnitude.
