  The use of trusted hardware has become a promising solution to enable
privacy-preserving machine learning. In particular, users can upload their
private data and models to a hardware-enforced trusted execution environment
(e.g. an enclave in Intel SGX-enabled CPUs) and run machine learning tasks in
it with confidentiality and integrity guaranteed. To improve performance, AI
accelerators have been widely employed for modern machine learning tasks.
However, how to protect privacy on an AI accelerator remains an open question.
To address this question, we propose a solution for efficient
privacy-preserving machine learning based on an unmodified trusted CPU and a
customized trusted AI accelerator. We carefully leverage cryptographic
primitives to establish trust and protect the channel between the CPU and the
accelerator. As a case study, we demonstrate our solution based on the
open-source versatile tensor accelerator. The result of evaluation shows that
the proposed solution provides efficient privacy-preserving machine learning at
a small design cost and moderate performance overhead.
