  Backdoor attack is a major threat to deep learning systems in safety-critical
scenarios, which aims to trigger misbehavior of neural network models under
attacker-controlled conditions. However, most backdoor attacks have to modify
the neural network models through training with poisoned data and/or direct
model editing, which leads to a common but false belief that backdoor attack
can be easily avoided by properly protecting the model. In this paper, we show
that backdoor attacks can be achieved without any model modification. Instead
of injecting backdoor logic into the training data or the model, we propose to
place a carefully-designed patch (namely backdoor patch) in front of the
camera, which is fed into the model together with the input images. The patch
can be trained to behave normally at most of the time, while producing wrong
prediction when the input image contains an attacker-controlled trigger object.
Our main techniques include an effective training method to generate the
backdoor patch and a digital-physical transformation modeling method to enhance
the feasibility of the patch in real deployments. Extensive experiments show
that PatchBackdoor can be applied to common deep learning models (VGG,
MobileNet, ResNet) with an attack success rate of 93% to 99% on classification
tasks. Moreover, we implement PatchBackdoor in real-world scenarios and show
that the attack is still threatening.
