  Physical quantities in the mixed $p$-spin glasses are evaluated with
Nishimori's gauge theory and several variance inequalities. The $\mathbb
Z_2$-symmetry breaking and the replica-symmetry breaking are studied in finite
and infinite dimensions. Obtained bounds on the expectation of the square of
the magnetization and spontaneous magnetization enable us to clarify properties
of paramagnetic and spin glass phases. It is proven that variances of
ferromagnetic and spin glass order parameters vanish on the Nishimori line in
the infinite volume limit. These results imply the self-averaging of these
order parameters on the Nishimori line. The self-averaging of the spin glass
order parameter rigorously justifies already argued absence of replica-symmetry
breaking on the Nishimori line.
