  Fully Homomorphic Encryption (FHE) refers to a set of encryption schemes that
allow computations to be applied directly on encrypted data without requiring a
secret key. This enables novel application scenarios where a client can safely
offload storage and computation to a third-party cloud provider without having
to trust the software and the hardware vendors with the decryption keys. Recent
advances in both FHE schemes and implementations have moved such applications
from theoretical possibilities into the realm of practicalities.
  This paper proposes a compact and well-reasoned interface called the
Homomorphic Instruction Set Architecture (HISA) for developing FHE
applications. Just as the hardware ISA interface enabled hardware advances to
proceed independent of software advances in the compiler and language runtimes,
HISA decouples compiler optimizations and runtimes for supporting FHE
applications from advancements in the underlying FHE schemes.
  This paper demonstrates the capabilities of HISA by building an end-to-end
software stack for evaluating neural network models on encrypted data. Our
stack includes an end-to-end compiler, runtime, and a set of optimizations. Our
approach shows generated code, on a set of popular neural network
architectures, is faster than hand-optimized implementations.
