  The continued growth of the computational capability of throughput processors
has made throughput processors the platform of choice for a wide variety of
high performance computing applications. Graphics Processing Units (GPUs) are a
prime example of throughput processors that can deliver high performance for
applications ranging from typical graphics applications to general-purpose data
parallel (GPGPU) applications. However, this success has been accompanied by
new performance bottlenecks throughout the memory hierarchy of GPU-based
systems. We identify and eliminate performance bottlenecks caused by major
sources of interference throughout the memory hierarchy.
  We introduce changes to the memory hierarchy for systems with GPUs that allow
the memory hierarchy to be aware of both CPU and GPU applications'
characteristics. We introduce mechanisms to dynamically analyze different
applications' characteristics and propose four major changes throughout the
memory hierarchy. We propose changes to the cache management and memory
scheduling mechanisms to mitigate intra-application interference in GPGPU
applications. We propose changes to the memory controller design and its
scheduling policy to mitigate inter-application interference in heterogeneous
CPU-GPU systems. We redesign the MMU and the memory hierarchy in GPUs to be
aware of ddress-translation data in order to mitigate the inter-address-space
interference. We introduce a hardware-software cooperative technique that
modifies the memory allocation policy to enable large page support in order to
further reduce the inter-address-space interference at the shared Translation
Lookaside Buffer (TLB). Our evaluations show that the GPU-aware cache and
memory management techniques proposed in this dissertation are effective at
mitigating the interference caused by GPUs on current and future GPU-based
systems.
