  Deep neural networks (DNNs) have revolutionized the field of computer vision
like object detection with their unparalleled performance. However, existing
research has shown that DNNs are vulnerable to adversarial attacks. In the
physical world, an adversary could exploit adversarial patches to implement a
Hiding Attack (HA) which patches the target object to make it disappear from
the detector, and an Appearing Attack (AA) which fools the detector into
misclassifying the patch as a specific object. Recently, many defense methods
for detectors have been proposed to mitigate the potential threats of
adversarial patches. However, such methods still have limitations in
generalization, robustness and efficiency. Most defenses are only effective
against the HA, leaving the detector vulnerable to the AA.
  In this paper, we propose \textit{NutNet}, an innovative model for detecting
adversarial patches, with high generalization, robustness and efficiency. With
experiments for six detectors including YOLOv2-v4, SSD, Faster RCNN and DETR on
both digital and physical domains, the results show that our proposed method
can effectively defend against both the HA and AA, with only 0.4\% sacrifice of
the clean performance. We compare NutNet with four baseline defense methods for
detectors, and our method exhibits an average defense performance that is over
2.4 times and 4.7 times higher than existing approaches for HA and AA,
respectively. In addition, NutNet only increases the inference time by 8\%,
which can meet the real-time requirements of the detection systems. Demos of
NutNet are available at: \url{https://sites.google.com/view/nutnet}.
