  Deep neural networks are successfully used in various applications, but show
their vulnerability to adversarial examples. With the development of
adversarial patches, the feasibility of attacks in physical scenes increases,
and the defenses against patch attacks are urgently needed. However, defending
such adversarial patch attacks is still an unsolved problem. In this paper, we
analyse the properties of adversarial patches, and find that: on the one hand,
adversarial patches will lead to the appearance or contextual inconsistency in
the target objects; on the other hand, the patch region will show abnormal
changes on the high-level feature maps of the objects extracted by a backbone
network. Considering the above two points, we propose a novel defense method
based on a ``localizing and inpainting" mechanism to pre-process the input
examples. Specifically, we design an unified framework, where the ``localizing"
sub-network utilizes a two-branch structure to represent the above two aspects
to accurately detect the adversarial patch region in the image. For the
``inpainting" sub-network, it utilizes the surrounding contextual cues to
recover the original content covered by the adversarial patch. The quality of
inpainted images is also evaluated by measuring the appearance consistency and
the effects of adversarial attacks. These two sub-networks are then jointly
trained via an iterative optimization manner. In this way, the ``localizing"
and ``inpainting" modules can interact closely with each other, and thus learn
a better solution. A series of experiments versus traffic sign classification
and detection tasks are conducted to defend against various adversarial patch
attacks.
