  Homomorphic encryption (HE)---the ability to perform computation on encrypted
data---is an attractive remedy to increasing concerns about data privacy in
deep learning (DL). However, building DL models that operate on ciphertext is
currently labor-intensive and requires simultaneous expertise in DL,
cryptography, and software engineering. DL frameworks and recent advances in
graph compilers have greatly accelerated the training and deployment of DL
models to various computing platforms. We introduce nGraph-HE, an extension of
nGraph, Intel's DL graph compiler, which enables deployment of trained models
with popular frameworks such as TensorFlow while simply treating HE as another
hardware target. Our graph-compiler approach enables HE-aware optimizations--
implemented at compile-time, such as constant folding and HE-SIMD packing, and
at run-time, such as special value plaintext bypass. Furthermore, nGraph-HE
integrates with DL frameworks such as TensorFlow, enabling data scientists to
benchmark DL models with minimal overhead.
