  Abuse of large language models reveals high risks as large language models
are being deployed at an astonishing speed. It is important to protect the
model weights to avoid malicious usage that violates licenses of open-source
large language models. This paper proposes a novel watermarking strategy that
plants watermarks in the quantization process of large language models without
pre-defined triggers during inference. The watermark works when the model is
used in the fp32 mode and remains hidden when the model is quantized to int8,
in this way, the users can only inference the model without further supervised
fine-tuning of the model. We successfully plant the watermark into open-source
large language model weights including GPT-Neo and LLaMA. We hope our proposed
method can provide a potential direction for protecting model weights in the
era of large language model applications.
