  The backdoor or Trojan attack is a severe threat to deep neural networks
(DNNs). Researchers find that DNNs trained on benign data and settings can also
learn backdoor behaviors, which is known as the natural backdoor. Existing
works on anti-backdoor learning are based on weak observations that the
backdoor and benign behaviors can differentiate during training. An adaptive
attack with slow poisoning can bypass such defenses. Moreover, these methods
cannot defend natural backdoors. We found the fundamental differences between
backdoor-related neurons and benign neurons: backdoor-related neurons form a
hyperplane as the classification surface across input domains of all affected
labels. By further analyzing the training process and model architectures, we
found that piece-wise linear functions cause this hyperplane surface. In this
paper, we design a novel training method that forces the training to avoid
generating such hyperplanes and thus remove the injected backdoors. Our
extensive experiments on five datasets against five state-of-the-art attacks
and also benign training show that our method can outperform existing
state-of-the-art defenses. On average, the ASR (attack success rate) of the
models trained with NONE is 54.83 times lower than undefended models under
standard poisoning backdoor attack and 1.75 times lower under the natural
backdoor attack. Our code is available at
https://github.com/RU-System-Software-and-Security/NONE.
