  We present a GPU-accelerated version of a high-order discontinuous Galerkin
discretization of the unsteady incompressible Navier-Stokes equations. The
equations are discretized in time using a semi-implicit scheme with explicit
treatment of the nonlinear term and implicit treatment of the split Stokes
operators. The pressure system is solved with a conjugate gradient method
together with a fully GPU-accelerated multigrid preconditioner which is
designed to minimize memory requirements and to increase overall performance. A
semi-Lagrangian subcycling advection algorithm is used to shift the
computational load per timestep away from the pressure Poisson solve by
allowing larger timestep sizes in exchange for an increased number of advection
steps. Numerical results confirm we achieve the design order accuracy in time
and space. We optimize the performance of the most time-consuming kernels by
tuning the fine-grain parallelism, memory utilization, and maximizing
bandwidth. To assess overall performance we present an empirically calibrated
roofline performance model for a target GPU to explain the achieved efficiency.
We demonstrate that, in the most cases, the kernels used in the solver are
close to their empirically predicted roofline performance.
