  The Strong Lottery Ticket Hypothesis (SLTH) stipulates the existence of a
subnetwork within a sufficiently overparameterized (dense) neural network that
-- when initialized randomly and without any training -- achieves the accuracy
of a fully trained target network. Recent works by Da Cunha et. al 2022;
Burkholz 2022 demonstrate that the SLTH can be extended to translation
equivariant networks -- i.e. CNNs -- with the same level of overparametrization
as needed for the SLTs in dense networks. However, modern neural networks are
capable of incorporating more than just translation symmetry, and developing
general equivariant architectures such as rotation and permutation has been a
powerful design principle. In this paper, we generalize the SLTH to functions
that preserve the action of the group $G$ -- i.e. $G$-equivariant network --
and prove, with high probability, that one can approximate any $G$-equivariant
network of fixed width and depth by pruning a randomly initialized
overparametrized $G$-equivariant network to a $G$-equivariant subnetwork. We
further prove that our prescribed overparametrization scheme is optimal and
provides a lower bound on the number of effective parameters as a function of
the error tolerance. We develop our theory for a large range of groups,
including subgroups of the Euclidean $\text{E}(2)$ and Symmetric group $G \leq
\mathcal{S}_n$ -- allowing us to find SLTs for MLPs, CNNs,
$\text{E}(2)$-steerable CNNs, and permutation equivariant networks as specific
instantiations of our unified framework. Empirically, we verify our theory by
pruning overparametrized $\text{E}(2)$-steerable CNNs, $k$-order GNNs, and
message passing GNNs to match the performance of trained target networks.
