  Today's cloud vendors are competing to provide various offerings to simplify
and accelerate AI service deployment. However, cloud users always have concerns
about the confidentiality of their runtime data, which are supposed to be
processed on third-party's compute infrastructures. Information disclosure of
user-supplied data may jeopardize users' privacy and breach increasingly
stringent data protection regulations. In this paper, we systematically
investigate the life cycles of inference inputs in deep learning image
classification pipelines and understand how the information could be leaked.
Based on the discovered insights, we develop a Ternary Model Partitioning
mechanism and bring trusted execution environments to mitigate the identified
information leakages. Our research prototype consists of two co-operative
components: (1) Model Assessment Framework, a local model evaluation and
partitioning tool that assists cloud users in deployment preparation; (2)
Infenclave, an enclave-based model serving system for online confidential
inference in the cloud. We have conducted comprehensive security and
performance evaluation on three representative ImageNet-level deep learning
models with different network depths and architectural complexity. Our results
demonstrate the feasibility of launching confidential inference services in the
cloud with maximized confidentiality guarantees and low performance costs.
