  This paper presents performance results comparing MPI-based implementations
of the popular Conjugate Gradient (CG) method and several of its communication
hiding (or 'pipelined') variants. Pipelined CG methods are designed to
efficiently solve SPD linear systems on massively parallel distributed memory
hardware, and typically display significantly improved strong scaling compared
to classic CG. This increase in parallel performance is achieved by overlapping
the global reduction phase (MPI_Iallreduce) required to compute the inner
products in each iteration by (chiefly local) computational work such as the
matrix-vector product as well as other global communication. This work includes
a brief introduction to the deep pipelined CG method for readers that may be
unfamiliar with the specifics of the method. A brief overview of implementation
details provides the practical tools required for implementation of the
algorithm. Subsequently, easily reproducible strong scaling results on the US
Department of Energy (DoE) NERSC machine 'Cori' (Phase I - Haswell nodes) on up
to 1024 nodes with 16 MPI ranks per node are presented using an implementation
of p(l)-CG that is available in the open source PETSc library. Observations on
the staggering and overlap of the asynchronous, non-blocking global
communication phases with communication and computational kernels are drawn
from the experiments.
