  We point out that charge conjugation and coordinate reflection symmetries do
not commute with the center symmetry of $SU(N)$ YM theory when $N>2$. As a
result, for generic values of the $\theta$ angle, the group of discrete
zero-form symmetries of YM theory on e.g. the spacetime manifold
$\mathbb{R}^3\times S^1$ includes the dihedral group $D_{2N}$ which is
non-Abelian for $N>2$. At $\theta = \pi$, the non-Abelian factor in the
symmetry group is enhanced to $D_{4N}$ due to discrete 't Hooft anomaly
considerations. We illustrate these results in YM theory as well as in a simple
quantum mechanical model, where we study representation theory as a function of
$\theta$ angle.
