  GPUs offer massive compute parallelism and high-bandwidth memory accesses.
GPU database systems seek to exploit those capabilities to accelerate data
analytics. Although modern GPUs have more resources (e.g., higher DRAM
bandwidth) than ever before, judicious choices for query processing that avoid
wasteful resource allocations are still advantageous. Database systems can save
GPU runtime costs through just-enough resource allocation or improve query
throughput with concurrent query processing by leveraging new GPU capabilities,
such as Multi-Instance GPU (MIG).
  In this paper we do a cross-stack performance and resource utilization
analysis of five GPU database systems. We study both database-level and
micro-architectural aspects, and offer recommendations to database developers.
We also demonstrate how to use and extend the traditional roofline model to
identify GPU resource bottlenecks. This enables users to conduct what-if
analysis to forecast performance impact for different resource allocation or
the degree of concurrency. Our methodology addresses a key user pain point in
selecting optimal configurations by removing the need to do exhaustive testing
for a multitude of resource configurations.
