  This paper introduces MGX, a near-zero overhead memory protection scheme for
hardware accelerators. MGX minimizes the performance overhead of off-chip
memory encryption and integrity verification by exploiting the
application-specific properties of the accelerator execution. In particular,
accelerators tend to explicitly manage data movement between on-chip and
off-chip memories. Therefore, the general memory access pattern of an
accelerator can largely be determined for a given application. Exploiting these
characteristics, MGX generates version numbers used in memory encryption and
integrity verification using on-chip accelerator state rather than storing them
in the off-chip memory; it also customizes the granularity of the memory
protection to match the granularity used by the accelerator. To demonstrate the
efficacy of MGX, we present an in-depth study of MGX for DNN and graph
algorithms. Experimental results show that on average, MGX lowers the
performance overhead of memory protection from 28% and 33% to 4% and 5% for DNN
and graph processing accelerators in a wide range of benchmarks, respectively.
