  With the commencement of the exascale computing era, we realize that the
majority of the leadership supercomputers are heterogeneous and massively
parallel even on a single node with multiple co-processors such as GPUs and
multiple cores on each node. For example, ORNLs Summit accumulates six NVIDIA
Tesla V100s and 42 core IBM Power9s on each node. Synchronizing across all
these compute resources in a single node or even across multiple nodes is
prohibitively expensive. Hence it is necessary to develop and study
asynchronous algorithms that circumvent this issue of bulk-synchronous
computing for massive parallelism. In this study, we examine the asynchronous
version of the abstract Restricted Additive Schwarz method as a solver where we
do not explicitly synchronize, but allow for communication of the data between
the sub-domains to be completely asynchronous thereby removing the bulk
synchronous nature of the algorithm.
  We accomplish this by using the onesided RMA functions of the MPI standard.
We study the benefits of using such an asynchronous solver over its synchronous
counterpart on both multi-core architectures and on multiple GPUs. We also
study the communication patterns and local solvers and their effect on the
global solver. Finally, we show that this concept can render attractive runtime
benefits over the synchronous counterparts.
