  In recent years, it has become increasingly popular to accelerate numerical
simulations using Graphics Processing Unit (GPU)s. In multiphysics simulations,
the various combined methodologies may have distinctly different computational
characteristics. Therefore, the best-suited hardware architecture can differ
between the simulation components. Furthermore, not all coupled software
frameworks may support all hardware. These issues predestinate or even force
hybrid implementations, i.e., different simulation components running on
different hardware. We introduce a hybrid coupled fluid-particle implementation
with geometrically resolved particles. The simulation utilizes GPUs for the
fluid dynamics, whereas the particle simulation runs on Central Processing Unit
(CPU)s. We examine the performance of two contrasting cases of a fluidized bed
simulation on a heterogeneous supercomputer. The hybrid overhead (i.e., the
CPU-GPU communication) is negligible. The fluid simulation shows good
performance utilizing nearly the entire memory bandwidth. Still, the GPU run
time accounts for most of the total time. The parallel efficiency in a weak
scaling benchmark for 1024 A100 GPUs is up to 71%. Frequent CPU-CPU
communications occurring in the particle simulation are the leading cause of
the decrease in parallel efficiency. The results show that hybrid
implementations are promising for large-scale multiphysics simulations on
heterogeneous supercomputers.
