  This document is one of the deliverable reports created for the ESCAPE
project. ESCAPE stands for Energy-efficient Scalable Algorithms for Weather
Prediction at Exascale. The project develops world-class, extreme-scale
computing capabilities for European operational numerical weather prediction
and future climate models. This is done by identifying Weather & Climate dwarfs
which are key patterns in terms of computation and communication (in the spirit
of the Berkeley dwarfs). These dwarfs are then optimised for different hardware
architectures (single and multi-node) and alternative algorithms are explored.
Performance portability is addressed through the use of domain specific
languages.
  Here we summarize the work performed on optimizations of the dwarfs focusing
on CPU multi-nodes and multi-GPUs. We limit ourselves to a subset of the dwarf
configurations chosen by the consortium. Intra-node optimizations of the dwarfs
and energy-specific optimizations have been described in Deliverable D3.3. To
cover the important algorithmic motifs we picked dwarfs related to the
dynamical core as well as column physics. Specifically, we focused on the
formulation relevant to spectral codes like ECMWF's IFS code.
  The main findings of this report are: (a) Up-to 30% performance gain with CPU
based multi-node systems compared to optimized version of dwarfs from task 3.3
(see D3.3), (b) up to 10X performance gain on multiple GPUs from optimizations
to keep data resident on the GPU and enable fast inter-GPU communication
mechanisms, and (c) multi-GPU systems which feature a high-bandwidth all-to-all
interconnect topology with NVLink/NVSwitch hardware are particularly well
suited to the algorithms.
