  We present an adversarial framework to craft perturbations that mislead
classifiers by accounting for the image content and the semantics of the
labels. The proposed framework combines a structure loss and a semantic
adversarial loss in a multi-task objective function to train a fully
convolutional neural network. The structure loss helps generate perturbations
whose type and magnitude are defined by a target image processing filter. The
semantic adversarial loss considers groups of (semantic) labels to craft
perturbations that prevent the filtered image {from} being classified with a
label in the same group. We validate our framework with three different target
filters, namely detail enhancement, log transformation and gamma correction
filters; and evaluate the adversarially filtered images against three
classifiers, ResNet50, ResNet18 and AlexNet, pre-trained on ImageNet. We show
that the proposed framework generates filtered images with a high success rate,
robustness, and transferability to unseen classifiers. We also discuss
objective and subjective evaluations of the adversarial perturbations.
