  Image classification has significantly improved using deep learning. This is
mainly due to convolutional neural networks (CNNs) that are capable of learning
rich feature extractors from large datasets. However, most deep learning
classification methods are trained on clean images and are not robust when
handling noisy ones, even if a restoration preprocessing step is applied. While
novel methods address this problem, they rely on modified feature extractors
and thus necessitate retraining. We instead propose a method that can be
applied on a $pretrained$ classifier. Our method exploits a fidelity map
estimate that is fused into the internal representations of the feature
extractor, thereby guiding the attention of the network and making it more
robust to noisy data. We improve the noisy-image classification (NIC) results
by significantly large margins, especially at high noise levels, and come close
to the fully retrained approaches. Furthermore, as proof of concept, we show
that when using our oracle fidelity map we even outperform the fully retrained
methods, whether trained on noisy or restored images.
