  We establish relations between different characterizations of order in spin
glass models. We first prove that the broadening of the replica overlap
distribution indicated by a nonzero standard deviation of the replica overlap
$R^{1,2}$ implies the non-differentiability of the two-replica free energy with
respect to the replica coupling parameter $\lambda$. In $\mathbb Z_2$ invariant
models such as the standard Edwards-Anderson model, the non-differentiability
is equivalent to the spin glass order characterized by a nonzero
Edwards-Anderson order parameter. This generalization of Griffiths' theorem is
proved for any short-range spin glass models with classical bounded spins. We
also prove that the non-differentiability of the two-replica free energy
mentioned above implies replica symmetry breaking in the literal sense, i.e., a
spontaneous breakdown of the permutation symmetry in the model with three
replicas. This is a general result that applies to a large class of random spin
models, including long-range models such as the Sherrington-Kirkpatrick model
and the random energy model.
