  Using the Polyakov-loop effective action, we investigate the structure of
spontaneously broken $\mathbb{Z}_N$ symmetry in the deconfinement vacuum in the
SU($N$) Yang-Mills theory with finite $N$. First, we examine the Polyakov-loop
fluctuation around a $\mathbb{Z}_N$-broken vacuum and calculate the spatial
correlation of its phase variable. We show that the phase variable of the
Polyakov loop becomes a Nambu-Goldstone mode in the large-$N$ limit. Second, we
estimate the global vacuum-to-vacuum transition rate in a finite-volume domain
of the quark-gluon plasma. Based on our estimation, we state that some
threshold volume exists, a domain larger than which is stable, and vice versa.
Identifying the threshold as the lower bound of a stable center domain volume,
we find the typical volume scale of center domains.
