  In the last decade, deep learning algorithms have become very popular thanks
to the achieved performance in many machine learning and computer vision tasks.
However, most of the deep learning architectures are vulnerable to so called
adversarial examples. This questions the security of deep neural networks (DNN)
for many security- and trust-sensitive domains. The majority of the proposed
existing adversarial attacks are based on the differentiability of the DNN cost
function.Defence strategies are mostly based on machine learning and signal
processing principles that either try to detect-reject or filter out the
adversarial perturbations and completely neglect the classical cryptographic
component in the defence. In this work, we propose a new defence mechanism
based on the second Kerckhoffs's cryptographic principle which states that the
defence and classification algorithm are supposed to be known, but not the key.
To be compliant with the assumption that the attacker does not have access to
the secret key, we will primarily focus on a gray-box scenario and do not
address a white-box one. More particularly, we assume that the attacker does
not have direct access to the secret block, but (a) he completely knows the
system architecture, (b) he has access to the data used for training and
testing and (c) he can observe the output of the classifier for each given
input. We show empirically that our system is efficient against most famous
state-of-the-art attacks in black-box and gray-box scenarios.
