  Fully Homomorphic Encryption (FHE) allows one to outsource computation over
encrypted data to untrusted servers without worrying about data breaching.
Since FHE is known to be extremely computationally-intensive,
application-specific accelerators emerged as a powerful solution to narrow the
performance gap. Nonetheless, due to the increasing complexities in FHE schemes
per se and multi-scheme FHE algorithm designs in end-to-end privacy-preserving
tasks, existing FHE accelerators often face the challenges of low hardware
utilization rates and insufficient memory bandwidth. In this work, we present
APACHE, a layered near-memory computing hierarchy tailored for multi-scheme FHE
acceleration. By closely inspecting the data flow across different FHE schemes,
we propose a layered near-memory computing architecture with fine-grained
functional unit design to significantly enhance the utilization rates of both
computational resources and memory bandwidth. In addition, we propose a
multi-scheme operator compiler to efficiently schedule high-level FHE
computations across lower-level functional units. In the experiment, we
evaluate APACHE on various FHE applications, such as Lola MNIST, HELR,
fully-packed bootstrapping, and fully homomorphic processors. The results
illustrate that APACHE outperforms the state-of-the-art ASIC FHE accelerators
by 2.4x to 19.8x over a variety of operator and application benchmarks.
