  Deep neural networks are vulnerable to backdoor attacks, where an adversary
maliciously manipulates the model behavior through overlaying images with
special triggers. Existing backdoor defense methods often require accessing a
few validation data and model parameters, which are impractical in many
real-world applications, e.g., when the model is provided as a cloud service.
In this paper, we address the practical task of blind backdoor defense at test
time, in particular for black-box models. The true label of every test image
needs to be recovered on the fly from a suspicious model regardless of image
benignity. We focus on test-time image purification methods that incapacitate
possible triggers while keeping semantic contents intact. Due to diverse
trigger patterns and sizes, the heuristic trigger search in image space can be
unscalable. We circumvent such barrier by leveraging the strong reconstruction
power of generative models, and propose a framework of Blind Defense with
Masked AutoEncoder (BDMAE). It detects possible triggers in the token space
using image structural similarity and label consistency between the test image
and MAE restorations. The detection results are then refined by considering
trigger topology. Finally, we fuse MAE restorations adaptively into a purified
image for making prediction. Our approach is blind to the model architectures,
trigger patterns and image benignity. Extensive experiments under different
backdoor settings validate its effectiveness and generalizability. Code is
available at https://github.com/tsun/BDMAE.
