  Applications that require substantial computational resources today cannot
avoid the use of heavily parallel machines. Embracing the opportunities of
parallel computing and especially the possibilities provided by a new
generation of massively parallel accelerator devices such as GPUs, Intel's Xeon
Phi or even FPGAs enables applications and studies that are inaccessible to
serial programs. Here we outline the opportunities and challenges of massively
parallel computing for Monte Carlo simulations in statistical physics, with a
focus on the simulation of systems exhibiting phase transitions and critical
phenomena. This covers a range of canonical ensemble Markov chain techniques as
well as generalized ensembles such as multicanonical simulations and population
annealing. While the examples discussed are for simulations of spin systems,
many of the methods are more general and moderate modifications allow them to
be applied to other lattice and off-lattice problems including polymers and
particle systems. We discuss important algorithmic requirements for such highly
parallel simulations, such as the challenges of random-number generation for
such cases, and outline a number of general design principles for parallel
Monte Carlo codes to perform well.
