  We review several parallel tempering schemes and examine their main
ingredients for accuracy and efficiency. The present study covers two selection
methods of temperatures and several choices for the exchange of replicas,
including a recent novel all-pair exchange method. We compare the resulting
schemes and measure specific heat errors and efficiency using the
two-dimensional (2D) Ising model. Our tests suggest that, an earlier proposal
for using numbers of local moves related to the canonical correlation times is
one of the key ingredients for increasing efficiency, and protocols using
cluster algorithms are found to be very effective. Some of the protocols are
also tested for efficiency and ground state production in 3D spin glass models
where we find that, a simple nearest-neighbor approach using a local n-fold way
algorithm is the most effective. Finally, we present evidence that the
asymptotic limits of the ground state energy for the isotropic case and that of
an anisotropic case of the 3D spin-glass model are very close and may even
coincide.
