  Backdoor watermarking is a promising paradigm to protect the copyright of
deep neural network (DNN) models. In the existing works on this subject,
researchers have intensively focused on watermarking robustness, while the
concept of fidelity, which is concerned with the preservation of the model's
original functionality, has received less attention. In this paper, focusing on
deep image classification models, we show that the existing shared notion of
the sole measurement of learning accuracy is inadequate to characterize
backdoor fidelity. Meanwhile, we show that the analogous concept of embedding
distortion in multimedia watermarking, interpreted as the total weight loss
(TWL) in DNN backdoor watermarking, is also problematic for fidelity
measurement. To address this challenge, we propose the concept of deep
fidelity, which states that the backdoor watermarked DNN model should preserve
both the feature representation and decision boundary of the unwatermarked host
model. To achieve deep fidelity, we propose two loss functions termed
penultimate feature loss (PFL) and softmax probability-distribution loss (SPL)
to preserve feature representation, while the decision boundary is preserved by
the proposed fix last layer (FixLL) treatment, inspired by the recent discovery
that deep learning with a fixed classifier causes no loss of learning accuracy.
With the above designs, both embedding from scratch and fine-tuning strategies
are implemented to evaluate the deep fidelity of backdoor embedding, whose
advantages over the existing methods are verified via experiments using
ResNet18 for MNIST and CIFAR-10 classifications, and wide residual network
(i.e., WRN28_10) for CIFAR-100 task. PyTorch codes are available at
https://github.com/ghua-ac/dnn_watermark.
