  In this paper, we propose a novel method for protecting convolutional neural
network (CNN) models with a secret key set so that unauthorized users without
the correct key set cannot access trained models. The method enables us to
protect not only from copyright infringement but also the functionality of a
model from unauthorized access without any noticeable overhead. We introduce
three block-wise transformations with a secret key set to generate learnable
transformed images: pixel shuffling, negative/positive transformation, and FFX
encryption. Protected models are trained by using transformed images. The
results of experiments with the CIFAR and ImageNet datasets show that the
performance of a protected model was close to that of non-protected models when
the key set was correct, while the accuracy severely dropped when an incorrect
key set was given. The protected model was also demonstrated to be robust
against various attacks. Compared with the state-of-the-art model protection
with passports, the proposed method does not have any additional layers in the
network, and therefore, there is no overhead during training and inference
processes.
