  Universal adversarial perturbation (UAP), also known as image-agnostic
perturbation, is a fixed perturbation map that can fool the classifier with
high probabilities on arbitrary images, making it more practical for attacking
deep models in the real world. Previous UAP methods generate a scale-fixed and
texture-fixed perturbation map for all images, which ignores the multi-scale
objects in images and usually results in a low fooling ratio. Since the widely
used convolution neural networks tend to classify objects according to semantic
information stored in local textures, it seems a reasonable and intuitive way
to improve the UAP from the perspective of utilizing local contents
effectively. In this work, we find that the fooling ratios significantly
increase when we add a constraint to encourage a small-scale UAP map and repeat
it vertically and horizontally to fill the whole image domain. To this end, we
propose texture scale-constrained UAP (TSC-UAP), a simple yet effective UAP
enhancement method that automatically generates UAPs with category-specific
local textures that can fool deep models more easily. Through a low-cost
operation that restricts the texture scale, TSC-UAP achieves a considerable
improvement in the fooling ratio and attack transferability for both
data-dependent and data-free UAP methods. Experiments conducted on two
state-of-the-art UAP methods, eight popular CNN models and four classical
datasets show the remarkable performance of TSC-UAP.
