  Stencil computations are widely used in HPC applications. Today, many HPC
platforms use GPUs as accelerators. As a result, understanding how to perform
stencil computations fast on GPUs is important. While implementation strategies
for low-order stencils on GPUs have been well-studied in the literature, not
all of proposed enhancements work well for high-order stencils, such as those
used for seismic modeling. Furthermore, coping with boundary conditions often
requires different computational logic, which complicates efficient
exploitation of the thread-level parallelism on GPUs. In this paper, we study
high-order stencils and their unique characteristics on GPUs. We manually
crafted a collection of implementations of a 25-point seismic modeling stencil
in CUDA and related boundary conditions. We evaluate their code shapes, memory
hierarchy usage, data-fetching patterns, and other performance attributes. We
conducted an empirical evaluation of these stencils using several mature and
emerging tools and discuss our quantitative findings. Among our
implementations, we achieve twice the performance of a proprietary code
developed in C and mapped to GPUs using OpenACC. Additionally, several of our
implementations have excellent performance portability.
