  This paper discusses the global $Z(N)$ symmetry of finite-temperature,
$SU(N)$, pure Yang-Mills lattice gauge theory and the physics of the phase of
the Wilson line expectation value. In the high $T$ phase, $\langle L \rangle$
takes one of $N$ distinct values proportional to the $Nth$ roots of unity in
$Z(N)$, and the $Z(N)$ symmetry is broken. Only one of these is consistent with
the usual interpretation $\langle L \rangle = e^{-F/T}$. This relation should
be generalized to $\langle L \rangle = z e^{-F/T}$ with $z \in Z(N)$ so that it
is consistent with the negative or complex values. In the Hamiltonian
description, the {\em physical} variables are the group elements on the links
of the spatial lattice. In a Lagrangian formulation, there are also group
elements on links in the inverse-temperature direction from which the Wilson
line is constructed. These are unphysical, auxiliary variables introduced to
enforce the Gauss law constraints. The following results are obtained: The
relation $\langle L \rangle=ze^{-F/T}$ is derived. The value of $z \in Z(N)$ is
determined by the external field that is needed for the infinite-volume limit.
There is a single physical, high-temperature phase, which is the same for all
$z$. The global $Z(N)$ symmetry is not physical; it acts as the identity on all
physical states. In the Hamiltonian formulation, the high-temperature phase is
not distinguished by physical broken symmetry but rather by percolating flux.
