  We conduct a systematic study of backdoor vulnerabilities in normally trained
Deep Learning models. They are as dangerous as backdoors injected by data
poisoning because both can be equally exploited. We leverage 20 different types
of injected backdoor attacks in the literature as the guidance and study their
correspondences in normally trained models, which we call natural backdoor
vulnerabilities. We find that natural backdoors are widely existing, with most
injected backdoor attacks having natural correspondences. We categorize these
natural backdoors and propose a general detection framework. It finds 315
natural backdoors in the 56 normally trained models downloaded from the
Internet, covering all the different categories, while existing scanners
designed for injected backdoors can at most detect 65 backdoors. We also study
the root causes and defense of natural backdoors.
