  Performing neural network inference on encrypted data without decryption is
one popular method to enable privacy-preserving neural networks (PNet) as a
service. Compared with regular neural networks deployed for
machine-learning-as-a-service, PNet requires additional encoding, e.g.,
quantized-precision numbers, and polynomial activation. Encrypted input also
introduces novel challenges such as adversarial robustness and security. To the
best of our knowledge, we are the first to study questions including (i)
Whether PNet is more robust against adversarial inputs than regular neural
networks? (ii) How to design a robust PNet given the encrypted input without
decryption? We propose PNet-Attack to generate black-box adversarial examples
that can successfully attack PNet in both target and untarget manners. The
attack results show that PNet robustness against adversarial inputs needs to be
improved. This is not a trivial task because the PNet model owner does not have
access to the plaintext of the input values, which prevents the application of
existing detection and defense methods such as input tuning, model
normalization, and adversarial training. To tackle this challenge, we propose a
new fast and accurate noise insertion method, called RPNet, to design Robust
and Private Neural Networks. Our comprehensive experiments show that
PNet-Attack reduces at least $2.5\times$ queries than prior works. We
theoretically analyze our RPNet methods and demonstrate that RPNet can decrease
$\sim 91.88\%$ attack success rate.
