  Benchmarking is an important challenge in HPC, in particular, to be able to
tune the basic blocks of the software environment used by applications. The
communication library and distributed run-time environment are among the most
critical ones. In particular, many of the routines provided by communication
libraries can be adjusted using parameters such as buffer sizes and
communication algorithm. As a consequence, being able to measure accurately the
time taken by these routines is crucial in order to optimize them and achieve
the best performance. For instance, the SKaMPI library was designed to measure
the time taken by MPI routines, relying on MPI's two-sided communication model
to measure one-sided and two-sided peer-to-peer communication and collective
routines. In this paper, we discuss the benchmarking challenges specific to
OpenSHMEM's communication model, mainly to avoid inter-call pipelining and
overlapping when measuring the time taken by its routines. We extend SKaMPI for
OpenSHMEM for this purpose and demonstrate measurement algorithms that address
OpenSHMEM's communication model in practice. Scaling experiments are run on the
Summit platform to compare different benchmarking approaches on the SKaMPI
benchmark operations. These show the advantages of our techniques for more
accurate performance characterization.
