  This work introduces a novel approach to achieving architecture-agnostic
equivariance in deep learning, particularly addressing the limitations of
traditional equivariant architectures and the inefficiencies of the existing
architecture-agnostic methods. Building equivariant models using traditional
methods requires designing equivariant versions of existing models and training
them from scratch, a process that is both impractical and resource-intensive.
Canonicalization has emerged as a promising alternative for inducing
equivariance without altering model architecture, but it suffers from the need
for highly expressive and expensive equivariant networks to learn canonical
orientations accurately. We propose a new method that employs any
non-equivariant network for canonicalization. Our method uses contrastive
learning to efficiently learn a unique canonical orientation and offers more
flexibility for the choice of canonicalization network. We empirically
demonstrate that this approach outperforms existing methods in achieving
equivariance for large pretrained models and significantly speeds up the
canonicalization process, making it up to 2 times faster.
