  Large-scale pre-trained models (PTMs) such as BERT and GPT have achieved
great success in diverse fields. The typical paradigm is to pre-train a big
deep learning model on large-scale data sets, and then fine-tune the model on
small task-specific data sets for downstream tasks. Although PTMs have rapidly
progressed with wide real-world applications, they also pose significant risks
of potential attacks. Existing backdoor attacks or data poisoning methods often
build up the assumption that the attacker invades the computers of victims or
accesses the target data, which is challenging in real-world scenarios. In this
paper, we propose a novel framework for an invisible attack on PTMs with
enhanced MD5 collision. The key idea is to generate two equal-size models with
the same MD5 checksum by leveraging the MD5 chosen-prefix collision.
Afterwards, the two ``same" models will be deployed on public websites to
induce victims to download the poisoned model. Unlike conventional attacks on
deep learning models, this new attack is flexible, covert, and
model-independent. Additionally, we propose a simple defensive strategy for
recognizing the MD5 chosen-prefix collision and provide a theoretical
justification for its feasibility. We extensively validate the effectiveness
and stealthiness of our proposed attack and defensive method on different
models and data sets.
