  Cloud providers are extending support for trusted hardware primitives such as
Intel SGX. Simultaneously, the field of deep learning is seeing enormous
innovation as well as an increase in adoption. In this paper, we ask a timely
question: "Can third-party cloud services use Intel SGX enclaves to provide
practical, yet secure DNN Inference-as-a-service?" We first demonstrate that
DNN models executing inside enclaves are vulnerable to access pattern based
attacks. We show that by simply observing access patterns, an attacker can
classify encrypted inputs with 97% and 71% attack accuracy for MNIST and
CIFAR10 datasets on models trained to achieve 99% and 79% original accuracy
respectively. This motivates the need for PRIVADO, a system we have designed
for secure, easy-to-use, and performance efficient inference-as-a-service.
PRIVADO is input-oblivious: it transforms any deep learning framework that is
written in C/C++ to be free of input-dependent access patterns thus eliminating
the leakage. PRIVADO is fully-automated and has a low TCB: with zero developer
effort, given an ONNX description of a model, it generates compact and
enclave-compatible code which can be deployed on an SGX cloud platform. PRIVADO
incurs low performance overhead: we use PRIVADO with Torch framework and show
its overhead to be 17.18% on average on 11 different contemporary neural
networks.
