  Recent works found that deep neural networks (DNNs) can be fooled by
adversarial examples, which are crafted by adding adversarial noise on clean
inputs. The accuracy of DNNs on adversarial examples will decrease as the
magnitude of the adversarial noise increase. In this study, we show that DNNs
can be also fooled when the noise is very small under certain circumstances.
This new type of attack is called Amplification Trojan Attack (ATAttack).
Specifically, we use a trojan network to transform the inputs before sending
them to the target DNN. This trojan network serves as an amplifier to amplify
the inherent weakness of the target DNN. The target DNN, which is infected by
the trojan network, performs normally on clean data while being more vulnerable
to adversarial examples. Since it only transforms the inputs, the trojan
network can hide in DNN-based pipelines, e.g. by infecting the pre-processing
procedure of the inputs before sending them to the DNNs. This new type of
threat should be considered in developing safe DNNs.
