  Due to insufficient training data and the high computational cost to train a
deep neural network from scratch, transfer learning has been extensively used
in many deep-neural-network-based applications. A commonly used transfer
learning approach involves taking a part of a pre-trained model, adding a few
layers at the end, and re-training the new layers with a small dataset. This
approach, while efficient and widely used, imposes a security vulnerability
because the pre-trained model used in transfer learning is usually publicly
available, including to potential attackers. In this paper, we show that
without any additional knowledge other than the pre-trained model, an attacker
can launch an effective and efficient brute force attack that can craft
instances of input to trigger each target class with high confidence. We assume
that the attacker has no access to any target-specific information, including
samples from target classes, re-trained model, and probabilities assigned by
Softmax to each class, and thus making the attack target-agnostic. These
assumptions render all previous attack models inapplicable, to the best of our
knowledge. To evaluate the proposed attack, we perform a set of experiments on
face recognition and speech recognition tasks and show the effectiveness of the
attack. Our work reveals a fundamental security weakness of the Softmax layer
when used in transfer learning settings.
