  Local random circuits scramble efficiently and accordingly have a range of
applications in quantum information and quantum dynamics. With a global $U(1)$
charge however, the scrambling ability is reduced; for example, such random
circuits do not generate the entire group of number-conserving unitaries. We
establish two results using the statistical mechanics of $k$-fold replicated
circuits. First, we show that finite moments cannot distinguish the ensemble
that local random circuits generate from the Haar ensemble on the entire group
of number conserving unitaries. Specifically, the circuits form a $k_c$-design
with $k_c = O(L^d)$ for a system in $d$ spatial dimensions with linear
dimension $L$. Second, for $k < k_c$, the depth $\tau$ to converge to a
$k$-design scales as $\tau \gtrsim k L^{d+2}$. In contrast, without number
conservation $\tau \gtrsim k L^{d}$. The convergence of the circuit ensemble is
controlled by the low-energy properties of a frustration-free quantum
statistical model which spontaneously breaks $k$ $U(1)$ symmetries. The
associated Goldstone modes are gapless and lead to the predicted scaling of
$\tau$. Our variational bounds hold for arbitrary spatial and qudit dimensions;
we conjecture they are tight.
