  New hardware primitives such as Intel SGX secure a user-level process in
presence of an untrusted or compromised OS. Such "enclaved execution" systems
are vulnerable to several side-channels, one of which is the page fault
channel. In this paper, we show that the page fault side-channel has sufficient
channel capacity to extract bits of encryption keys from commodity
implementations of cryptographic routines in OpenSSL and Libgcrypt --- leaking
27% on average and up to 100% of the secret bits in many case-studies. To
mitigate this, we propose a software-only defense that masks page fault
patterns by determinising the program's memory access behavior. We show that
such a technique can be built into a compiler, and implement it for a subset of
C which is sufficient to handle the cryptographic routines we study. This
defense when implemented generically can have significant overhead of up to
4000X, but with help of developer-assisted compiler optimizations, the overhead
reduces to at most 29.22% in our case studies. Finally, we discuss scope for
hardware-assisted defenses, and show one solution that can reduce overheads to
6.77% with support from hardware changes.
