  The US Department of Energy launched the Exascale Computing Project (ECP) in
2016 as part of a coordinated effort to achieve the next generation of
high-performance computing (HPC) and to accelerate scientific discovery. The
Exascale Proxy Applications Project began within the ECP to: (1) improve the
quality of proxies created by the ECP (2) provide small, simplified codes which
share important features of large applications and (3) capture programming
methods and styles that drive requirements for compilers and other elements of
the tool chain. This article describes one Proxy Application (or "proxy app")
suite called IMEXLBM which is an open-source, self-contained code unit, with
minimal dependencies, that is capable of running on heterogeneous platforms
like those with graphic processing units (GPU) for accelerating the
calculation. In particular, we demonstrate functionality by solving a benchmark
problem in computational fluid dynamics (CFD) on the ThetaGPU machine at the
Argonne Leadership Computing Facility (ALCF). Our method makes use of a
domain-decomposition technique in conjunction with the message-passing
interface (MPI) standard for distributed memory systems. The OpenMP application
programming interface (API) is employed for shared-memory multi-processing and
offloading critical kernels to the device (i.e. GPU). We also verify our effort
by comparing data generated via CPU-only calculations with data generated with
CPU+GPU calculations. While we demonstrate efficacy for single-phase fluid
problems, the code-unit is designed to be versatile and enable new physical
models that can capture complex phenomena such as two-phase flow with interface
capture.
