  This work studies the task of poisoned sample detection for defending against
data poisoning based backdoor attacks. Its core challenge is finding a
generalizable and discriminative metric to distinguish between clean and
various types of poisoned samples (e.g., various triggers, various poisoning
ratios). Inspired by a common phenomenon in backdoor attacks that the
backdoored model tend to map significantly different poisoned and clean samples
within the target class to similar activation areas, we introduce a novel
perspective of the circular distribution of the gradients w.r.t. sample
activation, dubbed gradient circular distribution (GCD). And, we find two
interesting observations based on GCD. One is that the GCD of samples in the
target class is much more dispersed than that in the clean class. The other is
that in the GCD of target class, poisoned and clean samples are clearly
separated. Inspired by above two observations, we develop an innovative
three-stage poisoned sample detection approach, called Activation Gradient
based Poisoned sample Detection (AGPD). First, we calculate GCDs of all classes
from the model trained on the untrustworthy dataset. Then, we identify the
target class(es) based on the difference on GCD dispersion between target and
clean classes. Last, we filter out poisoned samples within the identified
target class(es) based on the clear separation between poisoned and clean
samples. Extensive experiments under various settings of backdoor attacks
demonstrate the superior detection performance of the proposed method to
existing poisoned detection approaches according to sample activation-based
metrics.
