  Group Equivariant CNNs (G-CNNs) have shown promising efficacy in various
tasks, owing to their ability to capture hierarchical features in an
equivariant manner. However, their equivariance is fixed to the symmetry of the
whole group, limiting adaptability to diverse partial symmetries in real-world
datasets, such as limited rotation symmetry of handwritten digit images and
limited color-shift symmetry of flower images. Recent efforts address this
limitation, one example being Partial G-CNN which restricts the output group
space of convolution layers to break full equivariance. However, such an
approach still fails to adjust equivariance levels across data. In this paper,
we propose a novel approach, Variational Partial G-CNN (VP G-CNN), to capture
varying levels of partial equivariance specific to each data instance. VP G-CNN
redesigns the distribution of the output group elements to be conditioned on
input data, leveraging variational inference to avoid overfitting. This enables
the model to adjust its equivariance levels according to the needs of
individual data points. Additionally, we address training instability inherent
in discrete group equivariance models by redesigning the reparametrizable
distribution. We demonstrate the effectiveness of VP G-CNN on both toy and
real-world datasets, including MNIST67-180, CIFAR10, ColorMNIST, and
Flowers102. Our results show robust performance, even in uncertainty metrics.
