  Group equivariance can overly constrain models if the symmetries in the group
differ from those observed in data. While common methods address this by
determining the appropriate level of symmetry at the dataset level, they are
limited to supervised settings and ignore scenarios in which multiple levels of
symmetry co-exist in the same dataset. In this paper, we propose a method able
to detect the level of symmetry of each input without the need for labels. Our
framework is general enough to accommodate different families of both
continuous and discrete symmetry distributions, such as arbitrary unimodal,
symmetric distributions and discrete groups. We validate the effectiveness of
our approach on synthetic datasets with different per-class levels of
symmetries, and demonstrate practical applications such as the detection of
out-of-distribution symmetries. Our code is publicly available at
https://github.com/aurban0/ssl-sym.
