  The application resource specification--a static specification of several
parameters such as the number of threads and the scratchpad memory usage per
thread block--forms a critical component of modern GPU programming models. This
specification determines the parallelism, and hence performance, of the
application during execution because the corresponding on-chip hardware
resources are allocated and managed based on this specification. This
tight-coupling between the software-provided resource specification and
resource management in hardware leads to significant challenges in programming
ease, portability, and performance. Zorua is a new resource virtualization
framework, that decouples the programmer-specified resource usage of a GPU
application from the actual allocation in the on-chip hardware resources. Zorua
enables this decoupling by virtualizing each resource transparently to the
programmer.
  We demonstrate that by providing the illusion of more resources than
physically available via controlled and coordinated virtualization, Zorua
offers several important benefits: (i) Programming Ease. Zorua eases the burden
on the programmer to provide code that is tuned to efficiently utilize the
physically available on-chip resources. (ii) Portability. Zorua alleviates the
necessity of re-tuning an application's resource usage when porting the
application across GPU generations. (iii) Performance. By dynamically
allocating resources and carefully oversubscribing them when necessary, Zorua
improves or retains the performance of applications that are already highly
tuned to best utilize the resources.
