  Quantum many-body systems with exact local gauge symmetries exhibit rich
out-of-equilibrium physics such as constrained dynamics and disorder-free
localization. In a joint submission [J. C. Halimeh and P. Hauke,
arXiv:2004.07248], we present evidence of \textit{staircase prethermalization}
in a $\mathrm{Z}_2$ lattice gauge theory subjected to a small breaking of gauge
invariance. Here, we consolidate this finding and the associated emergent
nonperturbative timescales analytically and numerically. By means of a Magnus
expansion, we demonstrate how exact resonances between different
gauge-invariant supersectors are the main reason behind the emergence of
staircase prethermalization. Furthermore, we showcase the robustness of our
conclusions against various initial conditions including different system
sizes, matter fillings, and gauge-invariance sectors, in addition to various
boundary conditions, such as different maximal on-site matter occupations. We
also elaborate on how our conclusions are unique to local-symmetry models and
why they break down in the case of global-symmetry breaking. We moreover extend
our results to $\mathrm{U}(1)$ lattice gauge theories, illustrating the
generality of our findings. Our work offers an analytic footing into the
constrained dynamics of lattice gauge theories and provides proof of a certain
intrinsic robustness of gauge-theory dynamics to errors in experimental
settings.
