  Although deep neural networks have made tremendous progress in the area of
multimedia representation, training neural models requires a large amount of
data and time. It is well-known that utilizing trained models as initial
weights often achieves lower training error than neural networks that are not
pre-trained. A fine-tuning step helps to reduce both the computational cost and
improve performance. Therefore, sharing trained models has been very important
for the rapid progress of research and development. In addition, trained models
could be important assets for the owner(s) who trained them, hence we regard
trained models as intellectual property. In this paper, we propose a digital
watermarking technology for ownership authorization of deep neural networks.
First, we formulate a new problem: embedding watermarks into deep neural
networks. We also define requirements, embedding situations, and attack types
on watermarking in deep neural networks. Second, we propose a general framework
for embedding a watermark in model parameters, using a parameter regularizer.
Our approach does not impair the performance of networks into which a watermark
is placed because the watermark is embedded while training the host network.
Finally, we perform comprehensive experiments to reveal the potential of
watermarking deep neural networks as the basis of this new research effort. We
show that our framework can embed a watermark during the training of a deep
neural network from scratch, and during fine-tuning and distilling, without
impairing its performance. The embedded watermark does not disappear even after
fine-tuning or parameter pruning; the watermark remains complete even after 65%
of parameters are pruned.
