  Local symmetric quantum circuits provide a simple framework to study the
dynamics and phases of complex quantum systems with conserved charges. However,
some of their basic properties have not yet been understood. Recently, it has
been shown that such quantum circuits only generate a restricted subset of
symmetric unitary transformations [I. Marvian, Nature Physics, 2022]. In this
paper, we consider circuits with 2-local SU(d)-invariant unitaries acting on
qudits, i.e., d-dimensional quantum systems. Our results reveal a significant
distinction between the cases of d = 2 and d>2. For qubits with SU(2) symmetry,
arbitrary global rotationally-invariant unitaries can be generated with 2-local
ones, up to relative phases between the subspaces corresponding to inequivalent
irreducible representations (irreps) of the symmetry, i.e., sectors with
different angular momenta. On the other hand, for d>2, in addition to similar
constraints on the relative phases between the irreps, locality also restricts
the generated unitaries inside these conserved subspaces. These constraints
impose conservation laws that hold for dynamics under 2-local SU(d)-invariant
unitaries, but are violated under general SU(d)-invariant unitaries. Based on
this result, we show that the distribution of unitaries generated by random
2-local SU(d)-invariant unitaries does not converge to the Haar measure over
the group of all SU(d)-invariant unitaries, and in fact, for d>2, is not even a
2-design for the Haar distribution.
