  We study $Z_N$ symmetry in $SU(N)$ gauge theories in the presence of matter
fields in the fundamental representation, by restricting the lattice partition
function integration to matter fields which are uniform in spatial directions
and gauge fields with vanishing spatial components. In this approximation the
gauge matter field interaction effectively reduces to a 1-dimensional gauged
chain. This makes analytical calculations of the matter field contribution to
the Polyakov loop free energy possible. We show that in the limit of large
number of temporal sites the explicit breaking of $Z_N$ symmetry in this free
energy vanishes, driven by dominance of the density of states. We argue that
the spatial links as well as the spatial modes of the matter fields determine
the boundaries separating regions where $Z_N$ symmetry is realised from rest of
the phase diagram.
