  Modern applications are increasingly advanced and complex, and inevitably
contain exploitable software bugs despite the ongoing efforts. The applications
today often involve processing of sensitive information. However, the lack of
privilege separation within the user space leaves sensitive application secret
such as cryptographic keys just as unprotected as a "hello world" string.
Cutting-edge hardware-supported security features are being introduced.
However, the features are often vendor-specific or lack compatibility with
older generations of the processors. The situation leaves developers with no
portable solution to incorporate protection for the sensitive application
component.
  We propose LOTRx86, a fundamental and portable approach for user space
privilege separation. Our approach creates a more privileged user execution
layer called PrivUser through harnessing the underused intermediate privilege
levels on the x86 architecture. The PrivUser memory space, a set of pages
within process address space that are inaccessible to user mode, is a safe
place for application secrets and routines that access them. We implement the
LOTRx86 ABI that exports the privilege-based, accessing the protected
application secret only requires a change in the privilege, eliminating the
need for costly remote procedure calls or change in address space.
  We evaluated our platform by developing a proof-of-concept LOTRx86-enabled
web server that employs our architecture to securely access its private key
during SSL connection and thereby mitigating the HeartBleed vulnerability by
design. We conducted a set of experiments including a performance measurement
on the PoC on both Intel and AMD PCs, and confirmed that LOTRx86 incurs only a
limited performance overhead.
