  It is shown micromagnetic and atomistic spin dynamics simulations can use
multiple GPUs in order to reduce computation time, but also to allow for a
larger simulation size than is possible on a single GPU. Whilst interactions
which depend on neighbouring spins, such as exchange interactions, may be
implemented efficiently by transferring data between GPUs using halo regions,
or alternatively using direct memory accesses, implementing the long-range
demagnetizing interaction is the main difficulty in achieving good performance
scaling, where the data transfer rate between GPUs is a significant bottleneck.
A multi-GPU convolution algorithm is developed here, which relies on single-GPU
FFTs executed in parallel. It is shown that even for micromagnetic simulations
where the demagnetizing interaction computation time dominates, good
performance scaling may be achieved, with speedup factors up to 1.8, 2.5, and
3.1, for 2, 3, and 4 GPUs respectively. The code developed here can be used for
any number of GPUs in parallel, with performance scaling strongly dependent on
inter-GPU data transfer rate and connection topology. This is further improved
in micromagnetic simulations which include a spin transport solver, obtaining
speedup factors up to 1.96, 2.8, and 3.7, for 2, 3, and 4 GPUs respectively.
The best case scenario is obtained for atomistic spin dynamics simulations,
where the demagnetizing interaction is implemented with spin-averaged cells.
Using a single workstation with 4 GPUs, it is shown atomistic spin dynamics
simulations with up to 1 billion spins, and atomistic Monte Carlo simulations
with up to 2 billion spins are possible, with a near-ideal performance scaling.
